The Development of Psychosocial Maturity in Young Children: A Measure for Evaluating Character Education Programs

The Relationship Questionnaire’s assessment of psychosocial maturity based on social perspective coordination was developed within the cognitive-developmental research tradition (Kohlberg, 1981, 1984; Loevinger, 1966; Piaget, 1932/1965; Selman, 1980), influenced by these researchers’ intellectual predecessors in social theory (Baldwin, 1902, Dewey, 1933/ 1964; Mead, 1934) and by Werner’s (1948) orthogenetic theory. Selman and colleagues operationalized three related constructs in a set of research methods and instruments (including reflective individual interviews, manual-based scoring systems, and observations of social interactions) to develop a “Relationship Framework” during a 30-year research program. The premise of this framework is that social competence ultimately rests on the capacity for forming close relationships with other people, and that capacity in turn is grounded in psychosocial maturity, an internal psychological development. Furthermore, to the extent that persons lack the capacity to coordinate social perspectives, which underlies this construct of psychosocial maturity, they will be more likely to engage in risky and/ or antisocial behavior. Psychosocial maturity as measured in the “4-12” Rel-Q has consistently shown negative associations with risk-taking behaviors such as fighting, alcohol and drug use, and criminal activity in older children and adolescents (Adalbjarnardottir, 2002; Schultz, Barr, & Selman, 2001; Schultz, Selman, & LaRusso, 2003).

This model of relationship awareness includes three social-cognitive constructs: (1) interpersonal understanding, the theoretical knowledge of the nature of relationships (Selman, 1979, 1980), (2) interpersonal skills, including both interpersonal negotiation strategies (needed to resolve conflict in order to make and maintain good relationships) and shared experience or relatedness skills (Selman & Schultz, 1990), and (3) awareness of the personal meaning of relationships, the intensity and quality of the emotional investment an individual is able to make in specific other persons (Levitt & Selman, 1993). Although the three social-cognitive constructs in the Relationship Framework share the developmental substructure of capacity for social perspective coordination, these competencies tap distinct aspects of psychosocial functioning. For example, the constructs of interpersonal negotiation and personal meaning awareness are more affect-laden and contextual than the more strictly cognitive interpersonal understanding construct.

The three psychosocial competencies develop along parallel tracks from immature (undifferentiated) to mature (differentiated and integrated) levels. Table 1 presents the Relationship Framework, showing how the levels of social perspective coordination provide the developmental foundation of the psychosocial competencies.¹ The age progression in the emergence of social perspective coordination, which maps onto structural-developmental assessments in the cognitive and moral domains, was established through the assessment of psychosocial maturity with probed hypothetical social dilemmas (cf. Selman, 1980).

This developmental approach accommodates different levels of maturity in each of the three psychosocial competencies at a given point in time and/or across contexts. A key underlying assumption of the relationship model is that persons do not consistently reason or act at the same developmental level in different interpersonal contexts; even within a single incident or context the developmental level of action strategies may cover a wide range (Fischer & Bidell, 1998), and these performance levels are often not consistent with the highest level of social operations in the person’s repertoire (competence). The mapping of gaps between competence and performance provides a contextual dimension to this theory of social competence, distinct from its historical origins in the structural-developmental work of Piaget (1983) and Kohlberg (1969). In this sense the approach is heavily influenced by Werner’s (1948) classic view of comparative mental development. Although individuals can function at different levels in the three psychosocial competencies in different contexts, mature psychosocial capacity is reflected in social functioning in an integrative way (i.e., without gaps) at higher levels, when functionally appropriate.

A study of the “4-12” version of the Rel-Q that explored its usefulness as a tool for evaluating school-based character education programs (Schultz, Selman, & LaRusso, 2003) found significant developmental change on mean psychosocial maturity across grades 4 through 12 (with the exception of a plateau across the middle school grades). The Rel-Q also showed sensitivity to group differences based on differential socialization: girls scored significantly higher than boys at each grade level, and there were also with significant differences among students of different social classes and among those reporting different levels of negative risk-taking behavior. Quantitative and qualitative data on social climate predicted differences in mean psychosocial maturity between schools. These findings led to the following research questions with regard to the picture-based version of the Rel-Q for grades kindergarten through third grade:

1. What are the psychometric properties of the K-3 Rel-Q?

   It is expected that psychosocial maturity or relationship awareness as measured by the Rel-Q will show significant developmental differences from kindergarten through third grade, both cross-sectionally and longitudinally, supporting the measure’s validity. This age progression is expected to be similar in both geographically diverse samples. Gender differences in psychosocial maturity in this age range will also be examined, and differences among the individual psychosocial competencies (i.e., the Rel-Q subscales) will be explored.

2. What is the relation between the K-3 Rel-Q and teacher ratings of social skills?

   Psychosocial maturity is expected to be positively related to teacher ratings of students’ prosocial behaviors and negatively related to teacher ratings of problem behaviors. However, the effect size of these associations will likely be quite modest, since the ratings are not developmentally based.

3. Will the K-3 Rel-Q data fit the hypothesized structure of the measure, with three subscales and a general factor of psychosocial maturity or relationship awareness?

   A hierarchical confirmatory factor analysis will be conducted to examine this question.

This study used data derived from a classroom-based administration of the Rel-Q to children from kindergarten through third grade in two urban public school systems that were participating in character education and prevention programs.² Subjects from one Northeastern U.S. city, which totaled 2,046, including 62 kindergartners (age 5), 796 first graders (age 6), 784 second graders (age 7), and 404 third graders (age 8) from nine public schools, were tested in the fall of the school year. This sample was approximately half male and half female. A subsample of 200 first graders, 311 second graders, and 176 third graders in these schools in comparison classrooms not participating in the intervention were tested again in the spring, and this group was used in longitudinal (change score) analyses. This sample was 53% Black, 11% White, 5% Asian, and 31% Hispanic children. Seventy-four percent of the students were eligible to receive free or reduced-price meals.

A second sample from a Southern U.S. city was included to explore the representativeness of the Northeast sample. This sample of 2,030 students included a larger sample of kindergartners (475), 591 first graders (age 6), 445 second graders (age 7), and 519 third graders (age 8) from 6 public schools, tested in January of the school year. The gender of the subjects was not available for this sample. This sample was composed of 87% Black, 9% White, and 4% other races/nationalities. Eighty-seven percent of the students were eligible to receive free or reduced-price meals. As indicated by the free or reduced-price meal percentages, both samples consisted of relatively low social class populations.

The Rel-Q data reported were collected as part of program evaluations. In the Northeastern sample, although all students in the intervention classrooms participated in the character education programs, consent was required for participation in the evaluation. Written consent was obtained from parents and guardians through letters sent home and collected by their teachers. Only students who returned written permission sheets participated in the study; the participation rate ranged from 82% to 100%, with an average participation of 93%.³ All students in the Southern sample participated in the evaluation since the school system considered the questionnaire administration part of the program.

For part of the Northeastern sample (n = 393), teachers rated students’ social skills with the Teacher-Child Rating Scale (T-CRS) in both the fall and spring of the school year. The T-CRS (Hightower et al., 1986) was developed from two longer social skills scales, the Health Resource Inventory (Gesten, 1976) and Classroom Adjustment Rating Scale (Lorion, Cowen, & Caldwell, 1975) that were shown to have some psychometric shortcomings (Weissberg et al., 1987). The T-CRS instrument (Hightower et al., 1986) includes three problem scales (acting out, shy anxious, and learning difficulties) and three competence scales (educational task orientation, assertive social skills, and frustration tolerance). The three problem scales are derived from 18 items (e.g., “Is disruptive in class”) that teachers rate from 1 (not a problem) to 5 (very serious problem), and the three competence scales are derived from 20 items (e.g., “Is friendly toward peers”) that teachers rate as describing students from 1 (not at all) to 5 (very well). A validation study of the T-CRS showed high internal consistency (alphas ranging from .85 to .95), 10- and 20-week test-retest coefficients ranging from .61 to .91 with a median of .83, an ability to discriminate groups known to differ in adjustment, and convergent and divergent validity with other measures of child adjustment and performance in children from kindergarten through sixth grade.

The Relationship Questionnaire is a multiple-choice instrument that assesses the developmental level of the interpersonal competencies in the Relationship Framework. The measure focuses on relationships with both peers (friends and peer group acquaintances) and adults (parents and teachers). One version of the Rel-Q, like the majority of child self-report questionnaires, requires third to fourth-grade reading skills, and is appropriate for fourth grade through high school grade (the “4-12” version, Schultz, Selman, & LaRusso, 2003). The second version, which is the focus of this study, uses verbal instructions and pictures of animals instead of words and was specifically developed for younger children from kindergarten to third grade (the “K-3” version). The two conceptually similar, though empirically separate, versions have different questions with different multiple-choice responses, but are parallel in structure and based on the same constructs from the Relationship Framework.

The instruments were developed with data from qualitative interviews assessing interpersonal negotiation strategies and interpersonal understanding. Item prompts were constructed from the interview dilemmas, and multiple choice responses to these prompts representing different developmental levels were selected from actual subject answers.

Each multiple-choice response to the questions (or items) in the Rel-Q represents a particular developmental level of social perspective coordination in the domain (or scale) represented by the item. The two versions differ on the developmental range of the four responses to each item: whereas those on the “4-12” version range from impulsive to mutual, those on the “K-3” version range from impulsive to either reciprocal or in between reciprocal and mutual (see Table 1). The lower ceiling of the K-3 Rel-Q reflects the less complex social-cognitive capacities and social awareness of children who are transitioning from early to middle childhood.

The K-3 Rel-Q includes the following three scales, each with three items: interpersonal understanding of relationships, hypothetical interpersonal negotiation, and social perspective coordination. This picture-based version of the Rel-Q does not measure all three psychosocial competencies in the relationship model: there is no personal meaning awareness scale, in this version, though there is in the 4-12 Rel-Q.⁴ The K-3 Rel-Q has fewer subscales for related developmental reasons (the psychosocial competencies are less differentiated in early elementary children, partly because of their less developed cognitive ability, and the typical social problems that children of this age face tend to be less complex) as well as for methodological reasons (the animal picture-based format is less flexible than the written multiple-choice format).

The following are definitions and examples of items for the three Rel-Q scales on the K-3 version.

Interpersonal Understanding. Interpersonal understanding is defined as what the developing individual reflectively understands to be the core psychological and social qualities of persons and their relationships (Selman, 1980). Relationship understanding involves a child’s developing theoretical understanding of matters such as what goes into forming relationships or the conceptions of trust and jealousy in a close relationship, as distinct from the actions (strategies) that might be taken in the service of a relationship, the know-how of forming and maintaining a relationship. The following is one of the three interpersonal understanding items on the Rel-Q:

These four responses range from egocentric understanding (Horse saying he will always sit next to Lion in school) to a unilateral (first-person) understanding (Tiger giving Lion presents) to a response that is part unilateral, part reciprocal understanding (Elephant offering to always do what Lion tells him to do) to a fully reciprocal (second-person) understanding (Zebra saying he will never tell Lion’s secrets).

Interpersonal Negotiation. The developmental levels of interpersonal negotiation are also based on the core operation of social perspective coordination as we observe how a person resolves conflict with others in certain ways that indicate the extent to which the perspectives of self and other are taken into account and coordinated (Beardslee, Schultz, & Selman, 1987; Selman, Beardslee, Schultz, Krupa, & Podorefsky, 1986; Yeates, Schultz, & Selman, 1991). The autonomy-oriented interactions of interpersonal negotiation strategies are defined as the ways in which persons in situations of social conflict deal with the self and another person to gain control over inner and interpersonal disequilibrium. The following example is one of the three negotiation items on the Rel-Q that ask children to respond to a common early elementary peer conflicts:

The four responses here range from impulsive negotiation (Donkey pushing Bunny back out of line) to unilateral negotiation (Lion calling Bunny a cheater) to a transitional part unilateral, part reciprocal negotiation (Dove telling the teacher) to a reciprocal negotiation (Turtle telling Bunny that cutting in line is not fair).

Social Perspective Taking. Whereas interpersonal understanding items reflect the task of applying social perspective coordination to conceptualize relationships, and interpersonal negotiation items ask subjects to apply perspective coordination in the skill of responding to situations of social conflict, the social perspective taking items elicit a more pure social perspective coordination operation in response to a social problem involving multiple (though not conflicting) perspectives. These items tap operations that are similar to those of the social problem-solving research tradition (e.g., Spivak, Platt, & Shure, 1976). The following is one of the three perspective taking items on the Rel-Q:

These four responses range from egocentric perspective taking (Gorilla decided to get Wolf a hand puppet because Gorilla likes puppets) to one-way perspective taking (Polar Bear getting Wolf a puzzle because she interpreted Wolf’s comment to mean that she didn’t want another Teddy Bear) to a transitional part one-way, part second person response (Buffalo) to second-person perspective taking (Seal thought Wolf’s parents would know what she would want and deciding to ask them for a suggestion).

For each question (or item), there are four multiple-choice responses, each of which represents a particular level in the coordination of social perspectives. Unlike a typical multiple-choice questionnaire, Rel-Q subjects are asked not only to choose the “best” response, but also to rate each response as “poor,” “OK,” “good” or “excellent.” The combination of subjects rating each response and the use of pictures of animals results in a method of assessing the social competence capacity of younger children in a way that minimizes the impediments of their limited conceptual and literacy abilities. The process of successively rating each response breaks the task down for young children so that they are better prepared to pick their “best choice.” The methodology also adds four additional pieces of data per item, enhancing the psychometric properties of the measure. This multiple-rating method was key in solving problems inherent in the task of translating a complex qualitative/quantitative interview methodology to a more purely quantitative questionnaire one.

Consider a 6-year-old child’s responses to the interpersonal negotiation question presented above. Figure 1 shows the picture-based answer sheet filled out by a child named Jenny for the Bunny Cuts in Line item presented earlier. As each of the four responses is read in turn, the children are asked to circle (or color in) one of the faces representing respectively a “bad” way to deal with Bunny’s cutting in line (frowning face), an “OK” way (neutral face), a “good” way (smiling face), or an “excellent” way (smiling face with star). Note that this response rating is not a forced choice in which subjects have to designate one response as poor, a second one as average, a third as good, and a fourth as excellent. Rather, subjects choose any response rating (bad, OK, good, excellent) for each of the four responses within the question. Jenny, for example, rated two responses as bad (b and c), one as excellent, one as good, and no response as average (OK). Finally, the children are asked to “Look at the row with all the animals in it at the bottom of the page. Circle the animal that did the best thing when Bunny cut the line,” and the four choices are read again (the meaning of “best” is left to the subject). Jenny chose response “d” (Turtle) as his best choice. This “best choice” operation is a forced choice similar to typical multiple-choice methodology.

The Rel-Q yields two scores for each scale: a “response-rating” score, based on the children’s separate ratings of each of the four multiple choice responses, and a “best-choice” score, based on which response of the four they choose as “best.”

The scoring guide presents the following information for item 4, Bunny Cuts in Line:

4. Bunny Cuts in Line (interpersonal negotiation/conflict resolution)

The numbers in bold to the right of each multiple choice response (letters a, b, c, d) represent the developmental level assigned to the response based on previous theoretical and empirical analysis of subjects’ responses to interpersonal negotiation dilemmas (Schultz, Yeates, & Selman, 1988; Selman & Schultz, 1990). In this item, as in all the items, the responses are not in developmental level order but rather are ordered randomly. The developmental level of the response a child chooses as “best” is the score assigned as the “best-choice” score for each question. The scoring guide also presents the level scores to be assigned for each multiple-choice response subjects rate on the Likert scale of poor (or bad), average (or OK), good, or excellent. The corresponding number in the table is the level assigned for that particular response rating.

Jenny’s choices for Bunny Cuts in Line will serve as an example to illustrate the scoring process. She selected the reciprocal response “d” as the best, so her “best-choice” score would be 2.0 for this item. Jenny’s rating of the egocentric response 4a as “good” overrated it (by our standards) and is given a score of 1, her rating of the unilateral/reciprocal response 4b as “poor” underrated the item and also receives a score of 1, her rating of the unilateral response 4c as “poor” underrated the response somewhat and is given a score of 1.5, and her rating of the reciprocal item 4d as “excellent” reflects good judgment in our system and is given a score of 2. Jenny’s response-rating score for item 4 is 1.38, computed by summing the four level scores and dividing by 4: (1 + 1 + 1.5 + 2) divided by 4, which equals 5.5 divided by 4. Note that if a subject fails to rate all four choices or responses, the rated responses are summed and divided by the number of ratings the child did make (i.e., 3, 2 or 1) to compute the response-rating score.

Figure 2 presents a four-by-four table showing how these level scores were derived. Scoring items rated from poor to excellent involves giving credit for the extent to which subjects recognize the maturity or immaturity of a given item. The amount of credit subjects get for a given response rating is based on the range of developmental levels in the item.⁵ For this item (question 4, Bunny Cuts in Line), the response levels range from Level 0 to Level 2. In this case, subjects are given a response-rating score of 2 when they rate the responses exactly according to perspective coordination theory, corresponding to the diagonal in Figure 2. Thus, a score of 2 is given for a particular rating when level 0 responses are rated as poor, level 1 responses as average, level 1.5 responses as good, and level 2 responses as excellent (e.g., Jenny’s rating of 4d). When a rating falls off the diagonal by 1 box, the response has been somewhat overrated or underrated (e.g., Jenny’s rating of 3c, which was a level 1 response, as poor), and it is given a response score of 1.5. If a rating is off the diagonal by 2 boxes, which is even more of an underrating or overrating (e.g., Jenny’s rating 3a, which was a level 0 response, as good), the subject receives a response score of 1. Finally, if a level 0 response is highly overrated as excellent or a level 2 response is highly underrated as poor, the response score is 0. As described earlier for Jenny’s choices, the response-rating score for a given question is the sum of the individual response scores divided by the number of response ratings.

“Best-choice” scores for each of the three subscales (social perspective coordination, interpersonal understanding, and interpersonal negotiation) are computed by averaging the best-choice scores for each of the three items in that domain. Similarly, the three “response-rating” subscale scores are computed by averaging the three response-rating scores in a given subscale domain. The overall best-choice and response-rating scores, which represent two alternative ways to measure psychosocial maturity, are computed by averaging the three respective subscale scores. Because the best-choice and response-rating scores each have a developmental level metric (though the response-rating developmental level derivation is much less direct and obvious that that of the best-choice scores), the two scores can be averaged into an overall composite score. Descriptive statistics for all three scores—the best choice, response rating, and composite—are reported here, though the statistical analyses were conducted on the composite score only.

Table 2 presents Cronbach alphas for overall psychosocial maturity and the three subscales for both samples. The alpha for the composite form of overall psychosocial maturity was .75 in both samples, indicating moderately high internal consistency. The response-rating alpha was somewhat lower (.66 in both samples), and that of the best choice was much lower (.48 in one sample and .51 in the other), indicating only moderate internal consistency. The internal consistency scores for the subscales were very much lower, with alphas ranging from .70 to .53 for the composite score, from .62 to .32 for the response-rating score, and from .36 to .11 for the best-choice score. Of the three subscales, the interpersonal understanding scale had the strongest alphas. The alphas were very consistent across the two samples for each scale.

Table 3 presents ANOVAs across grades and gender for the Northeastern sample and across grades only for the Southern sample (for which gender was not available), using the composite Rel-Q score. The ANOVA for the Northeastern sample, which was conducted on the spring administration of the Rel-Q at each grade level, indicated significant grade effects from early spring of kindergarten through the end of third grade for psychosocial maturity and all three Rel-Q subscales. The F statistics for gender, which represent the effect of gender controlling for age (gender differences at each age), indicated that significant differences between girls and boys on psychosocial maturity and each subscale, with the largest gender difference in interpersonal negotiation and the smallest, though still significant, gender difference in perspective taking. There were no interactions between grade and gender. In the Southern sample ANOVA, grade also significantly predicted psychosocial maturity and each subscale at the p < .001. level. For both samples, the grade effect was especially large for interpersonal understanding.

Table 4 presents means and standard deviations for psychosocial maturity and the three subscales (using the composite score) at seven time points (both cross-sectional and longitudinal) from early spring of kindergarten through the end of third grade for the Northeastern sample and at each of the four (cross-sectional) grade levels for the Southern sample. The mean psychosocial maturity score rises progressively across these grades, from 1.32 in kindergarten to 1.68 in the spring of third grade.

Table 5 shows Scheffe multiple comparison tests conducted with ANOVAs on these composite scores to pinpoint where across the cross-sectional and longitudinal testing points the significant grade effects occurred. In the Northeastern sample, although there was no difference across the third grade year for psychosocial maturity, there were significant differences between each of the other consecutive testing points, with the exception of the short intervals (the summers) between school years (i.e., between the end of first grade and the beginning of second grade and between the end of second grade and the beginning of third grade). That is, the Rel-Q measured significant differences in psychosocial maturity between early spring of kindergarten and fall of first grade, across first grade, and across second grade. The pattern was the same for the subscales except there was not significant change from early spring of kindergarten to fall of first grade for interpersonal understanding and perspective taking, across first grade for perspective taking, and across second grade for interpersonal negotiation. Thus, the multiple comparisons for this sample suggest that growth in these psychosocial competencies is most robust across kindergarten, and first, and second grade but seems to slow in third grade, perhaps reflecting some ceiling effect. Even so, the grade effects are consistently significant across the first half of this period (kindergarten through fall of grade 2) and the second half (fall of grade 2 through spring of grade 3).

In the Southern sample, there was significant cross-sectional change across all three years for overall psychosocial maturity and the interpersonal understanding subscale, indicating that children develop on these competencies from kindergarten to first grade, from first to second grade, and from second to third grade. There was also significant change across first and second grades for interpersonal negotiation and perspective taking, and although the change from kindergarten to first grade was not significant on the composite score for these competencies, it was significant for the best-choice score for interpersonal negotiation and for the response-rating score for perspective taking. Thus, in the Southern sample growth in psychosocial maturity was relatively steady from the middle of the kindergarten year through the middle of third grade.

Table 6 shows the means broken out by gender for the overall score and subscale scores across the seven (cross-sectional and longitudinal) time points from kindergarten through third grade for the Northeastern sample, showing that girls scored significantly higher than boys across these ages. To further explore the gender differences found in the ANOVA analyses, t tests were conducted on each set of scores and gender differences that were significant or approaching significance are indicated with the appropriate symbol. Consistent gender differences began appearing at the end of grade 1 in overall psychosocial maturity and interpersonal negotiation, and became markedly robust by the end of second grade, continuing through third grade.⁵ In each case, girls scored at higher developmental levels than boys. In the spring of second grade (when the sample size was much larger than in third grade and thus smaller size differences become significant), there were gender differences on each subscale.

Table 7 presents change scores for the Rel-Q scales across school years for the longitudinal portion of the Northeastern sample, along with t statistics from paired t tests indicating their significance. The whole sample showed significant change across the year on the overall psychosocial maturity scale as well as on all three subscales. The first, second, and third graders, when examined separately, each showed significant change on the overall scores and on the subscale scores. In each case change was largest for the interpersonal understanding subscale and lowest, though still significant, for the interpersonal negotiation subscale. This pattern supports the validity of the developmental properties of the measure, and suggests that the K-3 Rel-Q is sensitive to psychosocial growth across these early elementary years.

Also shown in Table 7, the teachers observed little change across the year in the nondevelopmental Teacher-Child Rating Scale variables (though the first-grade teachers reported a significant decrease in two problem scales, the shy/anxious and learning problem scales). Table 8 compares the Rel-Q and Teacher-Child Rating Scale (T-CRS) in both fall and spring assessments for a subsample of second graders in the Northeastern sample. The correlations between psychosocial maturity and the prosocial T-CRS scales (easy-going behavior, assertive social skills, and task orientation) are consistently positive and significant, and those between psychosocial maturity and the teacher-rated problem behaviors (acting out, shy/anxious, and learning problems) are consistently negative and significant. These significant correlations, however, are at a small effect size (t’s ranging from an absolute value of .13 to .27). The modest size of these correlations between the K-3 Rel-Q and the T-CRS supports the conclusion that the Relationship Questionnaire seems to measure a related but quite different construct of social competence than that of the T-CRS, suggesting that phenomenological psychosocial maturity (or relationship awareness) is distinct from the types of social skills traditionally assessed as observed social competence.

The factor structure of the Rel-Q was explored using hierarchical confirmatory factor analysis in order to determine whether the hypothesized structure of the measure—with three first-order subscales (interpersonal understanding, interpersonal negotiation, and perspective taking) and one second-order factor (relationship maturity)—fit the data.

A nested series of hierarchical models were fitted on the nine response-rating and nine best-choice scores. These included three first-order models (one factor, three factor, and nine factor), two second-order models (one factor and three factor), and a third order model representing the hypothesized structure of the Rel-Q model (see Figure 3). As Christiansen, Lovejoy, Szymanski and Lang (1996) point out, comparisons of nested models like these provide a rigorous test of the structural validity of a measure beyond the information obtained from the overall goodness of fit of the hierarchical model itself. The use of nested model comparisons with hierarchical confirmatory factor analysis allows specific features of the Rel-Q model to be tested by restricting the solution in ways that systematically contrast competing higher order models and allow for a closer inspection of whether the factors of a given level are discriminable and justifiable (Rindskopf & Rose, 1988).

Table 9 presents the chi-square statistics, degrees of freedom, and fit statistics⁶ for the nested series of models and Table 10 presents a summary of the model comparisons. The null model, which posits that all of the observed variables are independent, fit the data poorly, supporting the proposed existence of common factors in subsequent models. The first-order models were compared to test whether the response-rating and best-choice scores for each of the nine items fit together (the nine factor model) or whether they independently loaded onto the three Rel-Q subscales (the three factor model). ⁷

To examine the discriminability of the first-order factors, the model positing a single factor (Model 1) was compared to the oblique (correlated) nine-factor model (Model 3b). As shown in Table 10, the one-factor model fit considerably worse than the oblique nine-factor models (Model 1 vs. Model 3b), suggesting that the 18 scores were poorly represented by a singular construct. A test of whether inclusion of the nine first-order factors in the model significantly improves fit (Model 2 vs. Model 5b/Model 6) further justified their structure and also suggests that they cannot be adequately represented by the three subscales as first-order factors.

Overall, the models tested justified the possibility of a hierarchical structure. Specifically, the model positing a single global factor (Model 1) fit the data much worse than the hierarchical model (Model 5b/6), indicating that the first and second-order unique factor variances were important to the model. Furthermore, the three-factor and nine-factor models allowing correlations among the factors (and therefore the potential for higher order factors) fit the data better than the orthogonal models, where no such structure was possible (Model 2a vs. Model 2b and Model 3a vs. Model 3b).

What is the viability of a second-order structure? An examination of the fit statistics show that Models 3b and 5b/6 both fit the data well, though they suggest that the fit of Model 3b is somewhat better, including the X2 (182 vs. 297), RMR (.007 vs. .008), RMSEA (.025 vs .035), and CFI (.980 vs. .958). However, these differences are slight and theory favors Model 5b/6. ⁸

Further comparisons explored the structure of the second-order factors and the potential for a third-order factor. The correlated factor model (Model 5b) fit the data significantly better than the orthogonal model (Model 5a), where there was no possibility of a higher order factor. Comparing Model 5b/6 to the model with a single second-order factor (Model 4) to test whether the three subscale factors were discriminable, the model with three correlated second-order factors fit the significantly better than the one-factor model. This result suggests that the three second-order factors are discrete and cannot be adequately represented by a more general factor.

From this series of nested model comparisons we conclude that the data fit the hypothesized structure of the Rel-Q measure, with nine first-order factors (i.e., representing the best-choice and response-rating scores for each item), three second-order factors (the three subscales), and a third-order factor (psychosocial maturity). Figure 4 illustrates the third-order model with the fitted parameters, showing high correlations among the second-order factors that support the use of the third-order factor of psychosocial maturity.

One of the most important psychometric characteristics of a developmental assessment in childhood is consistent age differences: older subjects should score higher on average than younger subjects. The Relationship Questionnaire showed significant age differences on psychosocial maturity and the three subscales. Cross-sectionally, kindergartners assessed in early spring scored significantly below first graders assessed in early fall on overall psychosocial maturity and certain subscales; first graders assessed in the early fall of the school year scored significantly below first graders assessed in the late spring, who in turn scored significantly below second graders assessed in the late spring in the Northeastern sample. Although this pattern did not hold consistently through third grade—the third graders tested in late spring did not score higher in the cross-sectional sample than those tested in the fall, change scores on the Rel-Q scales across the school year in the smaller longitudinal sample were highly significant for first, second, and third graders. Moreover, in the (cross-sectional) Southern sample, in which students were tested in January of the school year, there was significant change across each year between kindergarten and third grade. Not surprisingly, there were no differences between subjects assessed in the late spring of first grade and the early fall of second grade in the Northeastern sample—nor across the summer between second and third grade—since these assessments were only about four months apart.

Thus, although students in the kindergarten through third grade scored on average between the unilateral and reciprocal levels, students move from an average score close to the unilateral level in kindergarten to an average score closer to the reciprocal level by the end of third grade. The age progression in these cross-sectional and longitudinal data suggest that the Rel-Q is sensitive to developmental change in relationship maturity in the early elementary years. Although the evidence suggests that the natural growth in these psychosocial competencies is greater for the younger early elementary students, this may be an artifact of the measure—for example, some degree of ceiling effect may be operating for the third graders— since there was significant growth in psychosocial maturity across the later elementary years (fourth through sixth grade) on the reading-based version of the Rel-Q (cf. Schultz, Selman, & LaRusso, 2003).

Beginning at the end of first grade, girls’ psychosocial maturity was significantly higher than that of boys. Results from a study using the 4-12 Rel-Q suggest that this gender difference remains through 12th grade (Schultz, Selman, & LaRusso, 2003). Gender differences are typically found on measures of children’s social skills, with girls generally scoring as more socially adept than boys. For example, teachers rated girls higher than boys in kindergarten through sixth grade on the T-CRS (Hightower et al., 1986), except for the Shy/ Anxious scale. Similarly, teachers, parents, and students consistently gave better ratings to females at almost every grade level from preschool through 10th grade on the Social Skills Scale of the SSRS (Gresham & Elliott, 1990). Thus, relationship maturity as measured by the Rel-Q shows gender differences consistent with the T-CRS, SSRS, and other social skills measures. Despite some debate about whether gender differences in social development are due to biological or social processes, the finding that there are few gender differences in psychosocial maturity in kindergarten and early first grade, but that they emerge consistently at the end of first grade and continue through high school—and even into young adulthood (Schultz, Hauser, Selman & Allen, 2004)—suggest that these gender differences may be due to differential socialization, consistent with other evidence (Maccoby, 1990, 1998).

The multiple-choice methodology of the Rel-Q asks children to not only choose among four responses (best choice), but also to rate each of the four responses separately (response rating), a task in which children evaluate the responses according to their own developmental schema. This latter method is an innovative way to tap children’s social competence by comparing their rating of a range of social responses to a theoretical and empirical developmental ordering to convert the resulting score to a developmental metric.

The data with regard to internal consistency reliability provide less consistent support for the soundness of the Relationship Questionnaire than do the age and gender data, though this may speak to the complexity of the theoretical constructs rather than to deficient reliability of the measure. As argued in Schultz, Selman and LaRusso (2003), the higher reliability of the overall score reflects the deep structure of social perspective coordination on which the developmental levels of each of the psychosocial competencies (i.e., the subscales) are based; the lower reliability of the subscales reflects the contextual nature of these competencies, which theoretically are expected and empirically have been shown to vary across different interpersonal relationships and across different incidents or interactions within the same relationship (e.g., Selman et al., 1986). The higher estimated Cronbach alpha reliabilities on the 4-12 version of the Rel-Q (.85 for relationship maturity reported in Schultz, Selman & LaRusso, 2003) supports the notion that younger children are more labile in their psychosocial functioning than older children, both less consistent in coordinating social perspectives and more variable in applying it across contexts. The low internal consistency of the best-choice scores is due in part to the fact that these scores are based on only three data points for each of the three subscales, whereas the response-rating scores are based on 12 data points and the composite scores on 15 data points per subscale. Choosing the best choice represents a distinct cognitive task from that of rating each response, but because the reliability of the composite Rel-Q score is much more adequate than the response-rating score and (especially) the best-choice score, this score tends to be preferable in statistical analyses.

Hierarchical confirmatory factor analysis on a nested series of models confirmed that, as predicted by the theoretical model, a hierarchical model fits the Rel-Q data better than the one-factor model that neither pairs best-choice score and response-rating scores for each item nor groups the items into the three subscales of interpersonal understanding, interpersonal negotiation, and perspective coordination. Despite the relatively low internal consistency of the Rel-Q subscales, reflected in the “best” fit of the hierarchical model without the three subscales, the model that includes both the subscales and a higher-order general factor of relationship maturity also fits the data very well, providing support for the theoretical structure of the measure.

The psychosocial maturity of first, second, and third graders was positively correlated with teacher-rated T-CRS prosocial behaviors, as well as negatively correlated with teacher-rated problem behaviors, but at a small effect size. The teacher-rated social skills did not show the age progression of the Rel-Q scores. The T-CRS assessment seems based on observers’ norms of what constitutes culturally appropriate social behavior so that there is development in teacher’s (external) understanding of the child’s social behavior rather than in the cross-age scores. In contrast, the developmental standards are to a greater degree embedded in the expression of the child’s internal competencies on the Rel-Q. In rating children on social skills measures like the T-CRS, teachers may think good behavior consists of obedience and not causing difficulty, even if the child’s level of relational competence is low. On the other hand, children may have higher levels of social awareness but act up in class in ways that lower the teacher’s evaluation of their social adjustment. The very modest size of the correlation between the Rel-Q and the T-CRS suggests that the Rel-Q constructs overlap only somewhat with constructs of more traditional social skills measures, which represent culturally-driven, adult-centered evaluations of culturally-defined, appropriate behavior or social adjustment. The Rel-Q, in contrast, measures a related but quite different construct of developmentally-defined relationship awareness or psychosocial maturity that is sensitive to context.

There are a number of threats to the validity of the measure, as expected when assessing complex psychological processes with a multiple-choice instrument. Subjects can fill out the measure without thinking carefully about their responses, especially when it is administered in a group setting, such as a classroom. Although subjects can also be dismissive on an interview, the multiple-choice questionnaire format for assessing relationship maturity is much more subject to a lack of authentic engagement than an in-depth interview format. Though it may be less useful for individual assessment or diagnosis unless administered one-on-one, across large samples the measure successfully differentiates subjects in theoretically predicted ways.

The sample in this study was limited to urban, low social class populations because the data were collected in the course of the evaluations of prevention and character education programs in inner city schools. Research with the 4-12 version of the Rel-Q demonstrated that higher socioeconomic status (SES) suburban populations scored at higher levels of psychosocial maturity than lower SES urban populations; future research is needed to investigate whether this pattern would hold for the K-3 Rel-Q. Another limitation in the present study is that the developmental properties of the measure were examined solely through its correlation with grade. Further studies should explore whether children who do better on the measure actually have more mature relationships. Past research with interview and observational measures of interpersonal development based on social perspective coordination has found that mature social perspective coordination capacity is necessary but not sufficient for mature social behavior: children with low psychosocial maturity (measured as both interpersonal understanding and interpersonal negotiation strategies) have uniformly low level social behavior, whereas those with high psychosocial maturity may or may not have higher level social functioning (Selman, 1980; Selman & Schultz, 1990).

The Rel-Q can help evaluate the effectiveness of character education programs that aim to promote fairness, reciprocity, and other moral issues because social perspective coordination and hence psychosocial competence is inherent in these issues: if students lack the capacity to take the perspectives of others, they will fail to treat them fairly and with respect. Schultz, Selman, and LaRusso (2003) found that psychosocial maturity as measured by the Rel-Q is significantly correlated with moral reasoning measured with the Defining Issues Test (Rest, 1986) and, as discussed earlier, negatively associated with risky and antisocial behaviors such as fighting and criminal acts, providing evidence that psychosocial development is an important protective factor and resiliency process in character development. The 4-12 Rel-Q has shown great promise for character education and violence prevention program evaluation—for example, in a study of the Facing History and Ourselves program (Schultz, Barr, & Selman, 2000). The K-3 Rel-Q offers researchers an assessment tool that can differentiate children from kindergarten through third grade on a crucial aspect of social development—their level of psychosocial maturity or relationship awareness, providing a valuable new resource for the evaluation of character education programs for younger elementary-age children, for whom few measures of social development exist.