Construct validity and measurement invariance of the Asian Family Characteristics Scale in the Thai population Open Access

Family is the first and the most immediate social environment for almost every individual. Research has shown that the characteristics of one's family and its functioning have a profound impact on individuals' development of various characteristics, attitudes, behaviors and mental health issues. Studies showed that family cohesion and a positive parent–child relationship have small- to medium-sized positive correlations with the child's conscientiousness and agreeableness, small- to medium-sized negative correlation with neuroticism, and weak or nonsignificant correlation with extraversion and openness to experience [1–3]. Individuals who have a harmonious relationship with their family members tend to report higher subjective well-being [4, 5] and life satisfaction [6, 7].

Growing up in a better functioning family is also related to lower psychological distress [8, 9] and reduced risk of developing depression as an adult [10]. Compared to adolescents and young adults from stressful and conflict-filled families, those from close and harmonious families show greater self-compassion, which in turn leads to lower depression and anxiety [11]. On the other hand, adolescents from dysfunctional families are more likely to suffer from depression [12] and anxiety disorder [13] when they grew up.

Given the crucial role of family characteristics in the development of various mental health-related variables, reliable and valid measures of family characteristics and functioning could help family therapists and mental health practitioners screen for families experiencing problems, identify the domain in which they suffer and evaluate the effectiveness of the treatments.

Existing measures of family characteristics or family functioning were developed mostly in Western cultures and used mainly with a clinical population. Some examples of the widely used scales [14] include the McMaster Family Assessment Device (FAD) [15], a 60-item self-report scale developed based on the McMaster Model of Family Functioning [16]. It measures six dimensions of family functions, namely, communication, problem-solving, role functioning, affective responsiveness, affective involvement and behavioral control. The 42-item Family Adaptability and Cohesion Evaluation Scale (FACES-IV) [17] was invented according to the Circumplex Model of Marital and Family Systems [18]. The scale measures three dimensions: family cohesion, flexibility and communication. The Family Environment Scale (FES) [19] is another commonly used measure. This 90-item scale assesses three dimensions, which are further divided into 10 subdimensions: family relationships (cohesion, expressiveness and conflict subscales), personal growth (independence, achievement orientation, intellectual-cultural orientation, active recreational orientation and moral–religious emphasis) and system maintenance (organization and control).

These measures of family characteristics have been translated into other languages. However, direct translation may not work equally well in every country. The dynamics and meanings of the same behaviors might differ for Asian families with a higher level of collectivism, power distance and interdependence [20]. For example, in a research project funded by Malaysia's Ministry of Health, Malaysian researchers translated the FES [19] into the Malay language. Despite meticulous translation procedures, data showed that five out of ten subscales had very low reliability: Cronbach's alpha was 0.10–0.45. In response to that unsatisfactory result, Musa and colleagues [21] set out to develop a new measure that would better capture the Malaysian family's characteristics.

Musa and colleagues [21] reviewed existing measures of family functioning. They identified dimensions that describe Malaysian families, such as togetherness, family cohesion, expression, communication, common activities, family dynamic, family structure, ability to overcome challenges, religious affiliation and cultural beliefs. They then generated 43 items that described or reflected these dimensions. After an exploratory factor analysis with varimax rotation, they dropped 13 items with poor factor loadings. The remaining 30 items fell into five factors with six items each, namely, Togetherness and Harmony (α = 0.67), Expression (α = 0.81), Relationship and Family dynamic (α = 0.71), Conflict (α = 0.74) and Religiosity and Traditional Practice (α = 0.77). The overall scale also had high internal consistency (α = 0.90).

The first four factors in the AFCS are comparable to other family functioning measures. Musa and colleagues [21] noted that Religiosity and Traditional Practice emerged as a separate factor in the AFCS because they played an essential role in shaping the characteristics, norms and lifestyles of Malaysian families. Given the shared cultural heritage and traditions among countries in the Asian region, it is likely that these dimensions in the AFCS would also be relevant for families in other Asian countries. Recently, the scale has been translated into English and Indonesian. Translating and validating the scale into other Asian languages would allow for future cross-cultural research comparing multiple Asian countries.

The present study aimed to establish the AFCS's psychometric properties in the Thai population. In Study 1, we tested the structural validity of the Thai AFCS. We also conducted a measurement invariance test to verify that the scale's structure and meaning are equivalent across Thai and Malay samples. Then we tested the convergent and discriminant validity of the Thai AFCS in Study 2.

We translated the AFCS into Thai and collected data from a Thai sample. We tested how well the factor structure specified by Musa and colleagues [21] fit the empirical data from the Thai sample using confirmatory factor analysis. We also collected data from a Malay sample with the original AFCS. Then we ran a measurement invariance test to verify that the AFCS had the same structure and meaning across the Thai and Malay samples.

The rule of thumb for a sample size required to perform factor analysis was 10 participants per scale item [22]. The AFCS has 30 items; thus, we needed a sample of at least 300 participants. We recruited at least 60 participants from each of the five age groups (i.e. 18–24, 25–34, 35–44, 45–54 and 55 years and older) to ensure generalizability. A total of 384 Thai and 500 Malay participants completed the questionnaire. The participants from both countries (65% of the Thai sample and 78% of the Malay sample) were mostly female. Most participants had a college degree (80% of the Thai sample and 73% of the Malay sample). Approximately half of the participants were married (Table 1).

The original Malay version of the AFCS has five components, and each one had six items. Specifically, the five components included Togetherness and Harmony (e.g. “We exercise and carry out activities together”), Expression (e.g. “We keep our own problems to ourselves”), Relationship and Family dynamic (e.g. “We usually help one another”), Conflict (e.g. “We often raise our voice when discussing”) and Religiosity and Traditional Practice (e.g. “We are taught to respect other religious and cultural beliefs”).

Two native Thai speakers proficient in English translated the English version of the AFCS (Table 2) into Thai. Another two translators translated the scale back into English. All four translators and a Thai language expert met and discussed to determine, item by item, whether the back translations retained the meaning of the original items. The item in the Thai version that led to the most accurate back translation was chosen. The Thai language expert reviewed each item and revised some items to make it sound natural in Thai while retaining the same meaning.

After providing their consent, Thai and Malay participants filled out either a paper-and-pencil or online questionnaire in their respective languages. They rated how well each statement described their family on a scale of 1 (not at all) to 4 (to the greatest extent).

We performed confirmatory factor analysis using weighted least square mean and variance adjusted (WLSMV) estimator on the data from the Thai sample. The measurement model was defined according to the original model [21]. The initial five-factor model did not fit the data well χ² = 1011.676, df = 395, χ²/df = 2.651, comparative fit index (CFI) = 0.921, Tucker Lewis index (TLI) = 0.913, standardized root mean square residual (SRMR) = 0.068, root mean square error of approximation (RMSEA) = 0.064 with 90% confidence interval (CI) [0.059, 0.069].

We made four rounds of adjustments to the model. First, we followed the original paper by letting AFCS06 and AFCS13 load on both Factor 1 (Togetherness and Harmony) and Factor 3 (Relationship and Family Dynamic). Second, we allowed the residual of AFCS08 and AFCS09 to correlate due to their related meaning (keeping problems and feelings to oneself). We let the residuals of AFCS26, AFCS28 and AFCS29, which are about religion and spirituality, correlate with each other. Finally, the residuals of AFCS07, AFCS23 and AFCS24 were allowed to correlate because they shared the same meaning about arguing. The final model (Table 2) fit the observed data well: χ² = 748.786, df = 386, χ²/df = 1.940, CFI = 0.953, TLI = 0.948, SRMR = 0.058, RMSEA = 0.049 with 90% CI [0.044, 0.055]. Every item had a significant factor loading (p's < 0.05) on its designated factor. The overall scale had good internal consistency (α = 0.892) as did each of the five factors (α's ranged from 0.699–0.777)

We followed a recent approach to testing measurement invariance for ordinal (Likert-type) items [23, 24]. First, we verified that the pattern of factor-item relationships was the same across the Thai and Malay samples (configural invariance). Then we tested the equality of item intercepts between the two samples (threshold invariance). Finally, we examined whether the factor loadings were equivalent across samples (metric or loading invariance). A measurement model is considered invariant at each level when $Δ$CFI and $Δ$TLI are higher than –0.01, $Δ$SRMR is less than 0.015 and $Δ$RMSEA is less than 0.01 compared to the model at the previous step [25].

We started by applying the final model derived from the CFA to both samples simultaneously. The overall fit indices suggested that the model had configural invariance (M1); χ² = 1959.799, df = 782, χ²/df = 2.506, CFI = 0.951, TLI = 0.945, SRMR = 0.056, RMSEA = 0.058. Then the item thresholds were constrained to be equal across samples (M2). The model fitted the data no worse than the configural invariance model (M2-M1). Therefore, the threshold invariance held. Then we set the factor loadings to be equivalent across samples (M3). The overall model fit was worse than the threshold invariance model (M3-M2), suggesting that the metric invariance did not hold. We found that the factor loading of one item (AFCS29) was positive in the Thai sample but was negative in the Malay sample. We lifted the equality constraint on this item loading. The fit indices of this partial metric invariance model (M4) were comparable to those of the threshold invariance model (M4-M2). Based on these results, we concluded that the AFCS had an equivalent structure and meaning across the Thai and Malay samples (Tables 3 and 4).

The confirmatory factor analysis showed that the measurement model proposed by Musa and colleagues' could be replicated in the Thai sample [21]. The invariance test provided further evidence that the Thai and Malay versions of AFCS had equivalent meaning and structure.

Nevertheless, the AFCS structure could be more clean-cut if we dropped AFCS06 and AFCS13 that cross-loaded on both the Togetherness and Harmony and Relationship and family dynamic factors. Their loading on another factor was even stronger than their loading on the intended factors. There was also one item (AFCS29) that might be interpreted differently by the Thai and Malay samples. Dropping these questionable items would significantly improve the AFCS measurement model.

Since the validation studies for the current version of AFCS are being carried out in other languages, we intend to wait for data from other countries before deciding which item to eliminate in future studies. Until then, we would use this full version of AFCS and continue with our validity test in the Thai sample.

The purpose of Study 2 was to establish the Thai AFCS construct validity by examining its convergent and discriminant validity through its relationships with variables in the nomological network [26]. Specifically, we demonstrated convergent validity through the presence of the correlations between the AFCS and theoretically related variables. Discriminant validity would be supported by the absence of the correlation between the AFCS and theoretically unrelated variables. Based on the literature review, we hypothesized that AFCS would have a positive correlation with the Chulalongkorn Family Inventory, which measures family functioning developed in the Thai language. The AFCS should also be positively associated with a positive mindset and personality dimensions, such as self-compassion [11], satisfaction with life [4–7], agreeableness and conscientiousness [1–3]. In contrast, it should be negatively correlated with neuroticism [1–3], depression, anxiety and stress [8–13]. On the other hand, the literature suggested that the AFCS would have a low or nonsignificant association with extraversion and openness to experience [1, 2].

Four hundred and ninety-five Thai adults were recruited and randomly divided into four subsamples. Participants filled out the Thai AFCS and one other scale, which differed from sample to sample. See Table 5 for participants' demographic information.

The Thai AFCS from Study 1 had 30 items, divided into five factors with six items each. Participants rated how well each item described their family on a scale of 1 (not at all) to 4 (to the greatest extent). Items in two negatively worded factors, i.e. Expression and Conflict, were reverse coded before we took the average of the 30 items to form the overall AFCS score.

The 36-item Chulalongkorn Family Inventory (α = 0.88) was developed in the Thai language [27] based on the McMaster Model of Family Functioning [16]. It has seven components, namely, Problem-solving (e.g. “Family members can help each other solve most of the problems that occur at home”), Communication (e.g. “Family members can talk to each other frankly”), Role (e.g. “Family members have household chores that they are responsible for”), Affective responsiveness (e.g. “Family members openly express both good and bad feelings to each other”), Affective involvement (e.g. “Family members will only be interested in you if that matters to them”), Behavioral control (e.g. “Family members can do something wrong without being punished”) and General function (e.g. “Family members get along well”). Participants rated each item on a scale of 1 (never true) to 4 (always true), and higher scores reflected healthy family functioning.

The Thai version of the Self-Compassion Scale (α = 0.88) [28] was translated from Neff's SCS [29]. There are six subscales (three positive subscales and three negative subscales), including Self-kindness (e.g. “I am kind to myself when I am experiencing suffering”), Common humanity (e.g. “I try to see my failings as part of the human condition”), Mindfulness (e.g. “When something upsets me I try to keep my emotions in balance”), Self-judgment (e.g. “When times are really difficult, I tend to be tough on myself”), Isolation (e.g. “When I am feeling down I tend to feel like most other people are probably happier than I am”) and Over-identification (e.g. “When something upsets me I get carried away with my feelings”). Participants reported the frequency at which they thought, felt or behaved according to each of the 26 items on a scale of 1 (almost never) to 5 (almost always).

We translated the DASS-21 [30] into Thai. There are three separate subscales, namely, Depression (e.g. “I could not seem to experience any positive emotion,” α = 0.87), Anxiety (e.g. “I felt scared without any good reason,” α = 0.73) and Stress (e.g. “I tend to over-react to situations,” α = 0.83). Participants reported how much each statement applied to them over the past week on a scale of 1 (did not apply to me at all) to 4 (applied to me very much or most of the time).

The Thai version of the IPIP-NEO was developed [31] based on the International Personality Item Pool-NEO [32]. It has 30 items covering five domains, namely, Extraversion (e.g. “Keep in the background (R),” α = 0.76), Conscientiousness (e.g. “Make plans and stick to them,” α = 0.73), Agreeableness (e.g. “Have a good word for everyone,” α = 0.37), Neuroticism (e.g. “Often feel blue,” α = 0.83) and Openness to experience (e.g. “Believe in the importance of art”). Participants rated how accurately each item described themselves on a scale of 1 (very inaccurate) to 5 (very accurate).

We used the Thai version of SWLS [34]. Participants rated each of the five items (e.g. “The conditions of my life are excellent”) on a scale of 1 (strongly disagree) to 5 (strongly agree).

The questionnaire was administered online as well as in the paper-and-pencil format. Participants (n = 495) were divided into four subsamples. Participants in all four samples responded to the AFCS. Sample 1 (n = 121) also responded to Chulalongkorn Family Inventory (CFI); Sample 2 (n = 110) responded to the SCS; Sample 3 (n = 111) responded to the Depression, Anxiety and Stress Scale (DASS-21); and Sample 4 (n = 153) responded to the International Personality Item Pool-NEO (IPIP-NEO) and the Satisfaction with Life Scale.

Consistent with our predictions, participants' overall AFCS score was positively correlated with the Chulalongkorn Family Inventory (r = 0.531, p < 0.001), self-compassion (r = 0.541, p < 0.001), agreeableness (r = 0.284, p < 0.001), conscientiousness (r = 0.394, p < 0.001) and satisfaction with life (r = 0.373, p < 0.001). In contrast, the AFCS score was negatively related to the neuroticism (r = −0.501, p < 0.001), depression (r = −0.324, p < 0.01), anxiety (r = −0.241, p < 0.05) and stress (r = −0.346, p < 0.001). The results provided evidence for the convergent validity of the AFCS.

On the other hand, the discriminant validity of the AFCS was demonstrated by its nonsignificant correlations with extraversion (r = 0.082, p > 0.05) and openness to experience (r = 0.090, p > 0.05), which were expected to have low or no association with the AFCS (see Table 6).

The result provided evidence for the convergent and discriminant validity of the Thai AFCS. Consistent with findings in the literature, the AFCS had significant positive and negative correlations with variables that should be related [1–13] and no correlation with variables that should be weakly related [1–3] to family characteristics and functioning.

This paper was an initial attempt to develop the AFCS that would be suitable for the population in Asian cultures. The AFCS was first developed in the Malay language. Through Study 1 and Study 2, we demonstrated the Thai version of AFCS's structural, convergent, discriminant validity, as well as its measurement invariance across the Thai and Malay populations. The AFCS's first four factors mostly captured interpersonal dynamics in a family, while the last factor focused on the engagement and compliance with tradition or system within a family. The AFCS's togetherness and harmony dimension seemed to cover the FES's [19] active recreational orientation and FAD's problem-solving [15]. Expression was equivalent to FAD's communication. Relationship and family dynamic was comparable to the FAD's effective responsiveness and affective involvement as well as cohesion in FACES-IV [17] and FES. Conflict was the same as FES's conflict dimension. Finally, religiosity and traditional practice were a combination of the FES's moral-religious emphasis, organization and control dimensions. Unlike the FES, factors related to personal growth did not emerge as a separate domain. Interestingly, AFCS items that seemed to reflect independence (e.g. “We solve problems on our own”), which is considered a sign of personal growth in FES, loaded on negative factors.

This factor solution suggested that the relevant family characteristic domains in Asian countries like Malaysia and Thailand may differ from Western cultures. The factors that emerged in the AFCS reflected that families in Eastern cultures value interdependence, relationship harmony, and respect for tradition and order while perceiving that independence had a somewhat negative connotation.

The AFCS significantly predicted various mental health-related variables in the Thai sample. The overall scale had stronger correlations with personalities and mental health-related variables than did any of the five subscales. This might be due to the higher reliability of the overall score (Table 2). The overall score will be useful for screening individuals with dysfunctional families and the risk of mental health problems. On the other hand, each subscale score could help therapists identify specific domains of family characteristics that need to be addressed in the treatment.

The strength of the AFCS lies in the fact that it is specifically designed for Asian cultures. The current study tested its factor structure in the Thai and Malay samples from various age groups. The initial evidence of measurement invariance between the Malay and the Thai versions contributed to a valid comparison in future cross-cultural research, at least between these two Asian populations. On the other hand, cultural specificity might be seen as limiting direct comparison with Western cultures. However, given the findings that some of the AFCS subscales were conceptually comparable to other Western measures of family characteristics while some items seemed to have a different meaning, most mental health practitioners whose work does not need such cross-cultural comparison between Western and Eastern clients would probably benefit more from measures tailored to the families in their cultures.

We plan to make further improvements such as editing some questionable items and developing a short form that will facilitate the scale uses in both research and practice contexts. More research is also underway to translate and validate the AFCS in other Asian countries. We hope that the resulting reliable and valid AFCS in other Asian languages would allow meaningful cross-cultural comparison among families in multiple Asian countries on the dimensions that are genuinely relevant to Asian cultures.

The Thai version of AFCS has good reliability, structural validity and measurement invariance across the Thai and Malay populations. Its construct validity was demonstrated through its relationships with theoretically related constructs (convergent validity) and nonsignificant relationships with theoretically unrelated constructs (discriminant validity). We hope to validate the AFCS in other Asian languages to establish it as a useful tool for Asian researchers and practitioners and to inspire more cross-cultural research on family functioning across Asian countries.

Conflict of Interest: None

This research is supported by a grant from the Faculty of Psychology, Chulalongkorn University.