Assessment centers in the virtual age: validity and fairness in gender and age

Personnel evaluation for selection, training, and development is vital to any organization. Assessment centers (AC) have a long history (e.g. Ansbacher, 1941; Taft, 1948; Vernon, 1950; Thornton & Byham, 1982) and nowadays are widely popular as many organizations use them for personnel evaluation and selection (Hausknecht & Heavey, 2017). Scholars have viewed ACs as a central mode of personnel evaluation since their inception over 100 years ago (Thornton & Rupp, 2006; Baczynska, Skoczeń, Thornton, & Chen, 2024). They have maintained their popularity because of their numerous strengths. Scholars have associated them with content validity, negligible adverse effects, and high criterion-related validity (e.g. performance prediction) (Vernon, 1950; Schmitt, Gooding, Noe, & Kirsch, 1984; Gaugler, Rosenthal, Thornton, & Bentson, 1987; Iles, 1992; Thornton & Rupp, 2006) and supported it as a valid selection method (Rupp, Thornton & Gibbons, 2008; Arthur, Woehr, & Maldegen, 2000). The AC technique has evolved with the emergence of online, digital, and virtual ACs – terms often used interchangeably in prior research (e.g. Howland, Rembisz, Wang-Jones, Heise, & Brown, 2015; Avni & Luria, 2022) – as part of the broader adoption of technology and digital processes driven by the fourth industrial revolution and accelerated by the pandemic. In this study, we will refer to them collectively as virtual assessment centers (Virtual ACs).

The advancement of digital assessment techniques in the professional world is necessary to keep up with fast-paced innovations. It requires research to evaluate the techniques’ robustness in sight of massive use in the future. On the other hand, the need to study the validity of these new assessment forms has become increasingly important in research. Early studies laid the groundwork by exploring the transition from paper-and-pencil tests to computer-based tests and from in-person interviews to remote interviews, highlighting the critical need for continued validation of these digital methodologies (Sackett & Lievens, 2008).

Based on these premises, this work had two goals. First, we sought to provide initial virtual ACs’ construct validity evidence regarding internal and external validity (Grimm & Widaman, 2012). We explored whether we could synthesize the AC results in an overall rating (Thornton & Gibbons, 2009) and whether the AC results align with the promotion to the middle/high management level.

Second, considering the persistent gender disparities in career progression (e.g. Murphy, Callander, Dohan, & Grandis, 2021) and the possible advantage of younger people in using technologies for personnel selection (Woods, Ahmed, Nikolaou, Costa, & Anderson, 2019), we aimed to assess and establish the methodology’s adverse impact and focus on gender and age differences by examining the statistical significance of disparities between groups and investigating the potential influence of lead assessors’ gender on the equity of the selection process. We shall contribute to the literature about digital selection, which needs scientific support, and provide indications for reliable and fair use of virtual ACs during digital selection procedures by exploring their characteristics.

People use assessment centers in a variety of settings, for example, businesses, governmental organizations, educational institutions, and armed forces (Armoneit, Schuler, & Hell, 2020) to assist organizations in certification of employees’ competencies, promotions, selection for supervisory or management positions, and identification of employees with talent or high potential. Lately, the dive into the digital era has affected ACs, as we witnessed online assessments surge (Woods, Ahmed, Nikolaou, Costa, & Anderson, 2019), such as online applications, psychometric testing, digital interviews, gamified assessments, and social media mining. Both candidates and organizations favor digital techniques due to the higher comfort of use compared to physical assessment forms and the lower financial and geographical constraints (Templer & Lange, 2008; Howland, Rembisz, Wang-Jones, Heise, & Brown, 2015; Woods, Ahmed, Nikolaou, Costa, & Anderson, 2019). The use of digital tools in ACs has marked a technique’s evolution. However, the field still needs a thorough exploration of this evolution, even though scholars have studied specific methods commonly used in ACs, such as tests and interviews, in their digital forms (e.g. Van Iddekinge, Raymark, Roth, & Payne, 2006; Sears, Zhang, Wiesner, Hackett, & Yuan, 2013). A limited body of research has recently referred to these forms of AC as virtual assessment centers (henceforth, virtual ACs), namely, several activities aimed at evaluating people for selection or development purposes within organizations (e.g. in-baskets, leaderless group discussions, management games) supported by a wide range of digital platforms and online tools (e.g. online social networks, e-learning systems, immersive technology, gamification, simulation and biodata (Howland, Rembisz, Wang-Jones, Heise, & Brown, 2015; Lara-Prieto & Niño-Juárez, 2021). Herewith, virtual ACs differ from face-to-face ACs in that they do not require assesses to be physically present in the same location.

According to Grimm and Widaman (2012), the embodiment of an instrument’s validity is construct validity, which is the tool’s general capacity to measure the construct it intends to measure. Deeper down, we may articulate the construct validity along two axes: (1) internal validity (including content-related validity, dimensionality, reliability, and invariance) refers to the internal relationships among the indicators constituting a tool; (2) external validity (including criterion-related, discriminant and convergent validities, along with interpretation and implications of a tool’s use), which pertains to the relations between a tool and external criteria.

Assessment centers owe their popularity to their validity. High consensus exists on how the composite score derived from an AC tends to be a valid reflection of the required job criteria, for example, future job performance (Hermelin, Lievens, & Robertson, 2007; Arthur & Day, 2011; Sackett, Shewach, & Keiser, 2017). Identifying ACs’ critical dimensions is one of the key guidelines mentioned in the Guidelines and Ethical Considerations of Assessment Center Operations (Rupp et al., 2015). This affirms the significance of AC’s content validity. Therefore, organizations consistently use content-related validation methods to meet professional requirements (Woehr & Arthur, 2003). Moreover, scholars attribute ACs to notable high criterion-related validity, predictive validity correlations ranging from 0.37 to 0.52 to work-related criteria (Dilchert & Ones, 2009; Rupp, Thornton & Gibbons, 2008; Thornton & Gibbons, 2009), and incremental validity (Dayan, Kasten, & Fox, 2002). This infers that ACs provide distinctive future performance-related insight (Heimann, Ingold, Lievens, Melchers, Keen, & Kleinmann, 2021). Moreover, Heimann et al. (2021) showed that ACs were better suited to provide insight into personality traits. However, there has been less promising support for the construct-related validity of ACs (Meriac, Hoffman, Woehr, & Fleisher, 2008).

As a relatively novel format, scholars have not extensively studied virtual ACs for their psychometric properties. However, evidence supports measurement equivalence between traditional and online assessments (Lievens & Harris, 2003). Avni and Luria (2022) compared conventional and virtual ACs and found similar levels of validity and reliability. This is in line with other research that supports the efficacy of digital assessments (Langer, König, & Krause, 2017; Woods, Ahmed, Nikolaou, Costa, & Anderson, 2019). There is also a gap in exploring the criterion-related validity, namely how well a measure directly measures the construct or target behavior (Rosenbaum, 1989) of ACs and their content (e.g. incidents and exercises) (Hoffman, Kennedy, LoPilato, Monahan, & Lance, 2015). Overall, there is room for further research exploring the validity of online assessments like virtual ACs. As we attempted to fill this gap, our first research question was:

Gender biases exist and have been a source of hindrance to objectivity in employment-related decisions (Heilman & Eagly, 2008). Indeed, data that drives decisions essentially comes from humans. This information source may lead to unfairness. Biases based on stereotypes can affect decision-making tools, leading to different assessment standards for candidates and employees based on their gender. More specifically, they may adversely impact women’s chances of being recruited or promoted (Hoyt, 2012). For instance, according to Moscatelli, Menegatti, Ellemers, Mariani, and Rubini (2020), competence is the main criterion against which ACs evaluate men. In contrast, women must show excellent performance in three criteria, i.e. competence, morality, and sociability to make selection decisions that are in their favor.

We focused on two predominant types of unfairness concerns, i.e. disparate impact and disparate treatment. Adverse impact refers to the diverse rate of selection in recruitment, promotion, and/or other pertinent employment-related decisions that impact a particular group adversely or negatively, which may regard race, religion, gender, or ethnic group (In Outtz, 2010). Adverse impact reflects unintentional discrimination, largely governed by subconscious stereotypes. In contrast, disparate treatment is conscious and intentional discrimination in decision-making directed at employees because of their characteristics (Zafar, Valera, Gomez Rodriguez, & Gummadi, 2017). According to Prati et al. (2019), a significant factor influencing adverse impact is the gender of those responsible for evaluating the candidates (i.e. assessors or selectors). Specifically, male selectors tend to favor and recruit female candidates, while female selectors remain neutral. Both male and female selectors appear to be guided by differing standards when it comes to the assessment of male and female applicants. To prevent organizations from being overtly discriminatory and to promote equal employment opportunities, organizations have operationalized adverse impact in the form of selection ratios, commonly referred to as the “four-fifths rule” or “80% rule.” This rule states that the selection rate for any minority group should be at least 80% of the selection rate for the majority group. If the selection rate for the minority group is less than 80% of that of the majority group, this may indicate an adverse impact, suggesting potential discrimination. However, the ultimate indicator of adverse impact is the difference between the concerned groups and whether that difference is statistically significant and justifiable. Given that mixed teams have a higher probability of delivering superior decisions and contextual and task performances (Van de Voorde, Paauwe, & Van Veldhoven, 2012; Paauwe, Guest, & Wright, 2013; Prati et al., 2019), it remains important to gauge whether the organization’s decision-making can discriminate or not, for instance through an adverse impact analysis. In this research, we focused on the employees’ gender and specifically assessed if the methodology utilized for selecting candidates for Group Talent works at the expense of the members of a certain gender.

Therefore, we hypothesized:

Integrating digital tools into ACs may disproportionately benefit younger assessees. Indeed, younger individuals tend to have higher familiarity and comfort with digital technologies, as they have grown up in an environment rich in such technologies. This familiarity can lead to better performance in selection procedures that utilize digital platforms (Woods, Ahmed, Nikolaou, Costa, & Anderson, 2019). Moreover, younger assesses are often more adaptable to new digital innovations and technological changes. This adaptability can be beneficial in digital selection methods that are continually evolving (Folger, Brosi, Stumpf-Wollersheim, & Welpe, 2021). Finally, proficiency in handling digital tools and platforms could also improve the performance of younger assessees. Their ease in navigating and utilizing digital assessment tools can lead to a more accurate representation of their skills and competencies during selection procedures (Newhouse & Njiru, 2009).

Therefore, we hypothesized:

Overall, the study aimed to clarify whether we can consider the virtual AC a sound technique for helping organizations reach their goals connected to people evaluation selection and development. Our research questions sought to explore whether virtual ACs evaluate people’s characteristics (internal validity), whether this evaluation is connected with relevant criteria (external validity) (RQ1), and whether virtual ACs are immune from adverse impact implications related to discriminating standards regarding gender (RQ2) or to the discrepant levels of familiarity between younger and older assesses (RQ3). In the following part, we present the methodology adopted to answer these RQs.

To answer our research questions, we employed a digital AC developed to identify the group talent from an Italian Group of Companies. We standardized the AC procedure and administered it to all the participants uniformly. The exercises and assessors remained unchanged for all the participants to the AC except for one particular exercise, namely in-basket or business case presentation, between the two participants’ levels (see 3.3. section). The final steps consisted of scoring and analysis between the assessors and the relevant organization members.

The sample consisted of 120 employees from an Italian Group of Companies. About 55.8% of the participants were men and 43.3% were women. Of them, 91 (75.8%) were managers, while 29 (24.2%) were senior managers. From the Italian Group of Companies, 16 participants belonged to Company 1 and 2 each, 4 participants were from Company 3, 13 were from Company 4, 4 participants belonged to Company 5, while Company 6 had 24, which was the greatest number of participants, 5 participants belonged to Company 7, and lastly, 24 participants were from the Corporate. The participants’ ages ranged from 28 years to 51 years, with an average age of 37.85 years (SD = 5.13 years). Moreover, there was a total of 15 lead assessors. Of them, 11 (73.3 %) were women, and 4 (26.7%) were men.

We selected the participants from each company’s internal human resources department in collaboration with the direct supervisor responsible for selecting the best performer. The selection criteria for the candidates partaking in the AC depended on two factors, i.e. fluency in English (minimum B2 level) and compliance with the Company values. The AC was an individually administered procedure conducted on digital platforms in English. It consisted of several exercises, namely the BEI interview, in-basket or business case presentation, and direct report meeting. We conducted them according to a standardized schedule, and they took about 4.5 hours to complete. We designed the framework around eight competencies derived from the Company’s Soft Skills Model, structured as follows:

1. Execution: the application of organizational principles, knowledge, and expert skills to deal with critical issues and achieve results;

2. Proactivity: the ability to anticipate business trends and act accordingly;

3. Decision-making: the ability to make informed choices with calculated risks after context assessment (organizational goals and local market’s understanding);

4. Adaptability: the capacity to adjust under varying circumstances and changing goals;

5. People management: the ability to create a learning environment, adopt effective communication, and customize content for an engaged workforce;

6. Teamwork: the ability to share and communicate knowledge for high-quality performance with colleagues;

7. Innovation: the ability to implement ideas backed by expertise and insights;

8. Openness and inclusion: the ability to include every member by ensuring open access to information and utilizing efficient communication.

The Lead Assessor initiated the AC with a presentation on Microsoft Teams and showed the agenda. Three stages followed after the meeting.

BEI interview: It consisted of a 75-min interview with the lead assessor. During the interview, the participants completed the Hogan personality test and cognitive test Matrigma or HRBI along with a Word document that included their profile summaries, including their education level and professional experience. We also generated insight into the candidates’ future aspirations to gauge motivation in the candidate.

In-basket/business case presentation: This stage presented an individual problem-solving exercise. It consisted of different levels of complexity in terms of the business cases presented to the participants according to their manager or senior manager role. The participants had 90 minutes to solve the business case individually and 30 minutes to present to the Lead Assessor without any discussion.

Direct report meeting: It consisted of an interactive exercise and was uniform for both managerial groups. It comprised a business simulation in which the participants faced a real-life situation experienced by a particular employee. The participants had a 40-min time limit, with 20 minutes to assess the situation and 20 minutes to discuss the situation with the second assessor.

Following the administration, we utilized the BARS method to score each exercise. We collected the scoring on an integration grid that we also utilized for the integration of results and its subsequent scoring on the algorithm that we prepared to tabulate four score levels: a total competency score, which was the average of the candidate’s score in each competency, a potential score which determined the potential level, an international attitude score which determined the extent to which a candidate was keen in being a part of different cultures, and global opportunities. We measured it by specific questions during the BEI interview. Those questions were designed to gauge past international experiences, the person’s ability to coherently elaborate on those experiences and their value, and the willingness to continue to invest in their global expertise. Lastly, we gauged the career aspiration score during the BEI interview to assess the level of ambition a candidate has regarding their professional growth. Together, all these scores determined the status validation of the participant as a Group Talent.

Furthermore, determining a participant’s designation as Group Talent entailed three distinct validation levels during its final phase. Initially, we held a meeting between the company’s human resources business partner and the lead assessors to review and integrate the assessment scores with historical data on the applicant’s performance. In the subsequent stage, we gathered the human resources business partner, lead assessors, and the participants’ direct supervisors, where we further scrutinized the results from the first meeting and combined them with the supervisors’ assessments. We made the ultimate decision in a board-level meeting attended by senior managers, senior HR executives (C-level), and a director, focusing on promotion decisions. Three key factors informed this comprehensive evaluation process, i.e. participants’ professional histories, their AC scores, and the evaluations provided by their supervisors. Thus, a holistic review resulted in the final identification of participants as Group Talent. However, despite being comprehensive, the AC’s validity criterion was not entirely independent since one-third relied on the AC scores.

We subjected the data obtained from the digital AC to a series of statistical analyses using IBM SPSS Statistics 27. We calculated the means, standard deviations, and explorative analyses to gauge the data distribution. To address the RQ1, we analyzed the following elements as indexes of the internal validity: (1) the dimensionality through the correlation between the competencies assessed during the virtual AC and through a principal component analysis; (2) reliability with Cronbach’s alpha and McDonald’s omega. Furthermore, we assessed external validity with criterion-related validity based on a binary logistic regression and sensitivity analysis (Grimm & Widaman, 2012).

To address RQ2, we calculated the Chi-squared test and the four-fifth rule to get an insight into the fairness of the digital AC as a selection tool specifically related to gender and to assess adverse impact. Finally, we adopted MANOVA to analyze the impact of gender (participant and assessor) in response to RQ2 and age in response to RQ3.

The first research question pertained to AC’s validity in its virtual format. Table 1 reports the mean and standard deviations of the competencies. The mean scores generally reside in the middle point of the scale, which is in line with the normal distribution. The standard deviations were medium level. The skewness values suggest distributions that are very close to symmetry. They were almost evenly balanced on both sides of the mean, with only subtle tendencies toward positive or negative skew. The kurtosis values in the range of 0.8 to −0.41 indicated a moderately platykurtic distribution.

Regarding internal validity, Table 2 shows the correlations between the competencies, specifically highlighting the highest and lowest correlations. Every competence was significantly correlated with each other, which implies that the dimensions fit to be a part of the virtual AC. Execution exhibited the strongest significant positive correlation with adaptability (r = 0.48, p < 0.05) and had the weakest yet significant positive correlation (r = 0.21, p < 0.01) with people management. Proactivity had the strongest positive correlations with adaptability (r = 0.48, p < 0.05) and innovation (r = 0.48, p < 0.05) and the weakest yet significant and positive correlation with decision-making (r = 0.35, p < 0.05). Decision-making showed the strongest and significantly positive correlation with execution (r = 0.44, p < 0.05) and a significantly positive but weakest correlation with people management (r = 0.21, p < 0.05). Adaptability exhibited the strongest significant positive correlation with teamwork (r = 0.60, p < 0.05) and exhibited the weakest yet significant positive correlation with decision-making (r = 0.41, p < 0.05). People management had the strongest significant positive correlation with openness and inclusion (r = 0.61, p < 0.05), whereas it had a significant yet weakest correlation with execution (r = 0.21, p < 0.01). Teamwork had the strongest significant positive correlation with openness and inclusion (r = 0.67, p < 0.05) and the weakest yet significant positive correlation with proactivity (r = 0.36, p < 0.05). Innovation had the strongest and most significant correlation with teamwork (r = 0.54, p < 0.05) and the weakest yet significant positive correlation with people management (r = 0.38, p < 0.05). Lastly, openness and inclusion exhibited the strongest significant positive correlation with teamwork (r = 0.67, p < 0.05) and the weakest yet significant positive correlation with execution (r = 0.38, p < 0.05).

Based on the eigenvalue greater than 1.0 rule, the scree plot, and the parallel analysis, the results of the principal component analysis showed the presence of one principal component. This component explains a significant 51.1% of the variance, which suggests that it captures a major underlying factor influencing all these attributes. Moreover, Bartlett’s test of sphericity was significant (χ²(28) < 0.001), indicating that the variables were related and suitable for PCA. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was high (0.859) and thus confirmed that the data was suitable for PCA. The measure of sampling adequacy (MSA) for individual items ranged from 0.832 to 0.903, which is considered good. The PCA results demonstrate that the component had significant loadings, ranging from 0.61 for decision-making to 0.81 for teamwork, indicating that these attributes contribute meaningfully to the primary component (Table 3).

The results support a unidimensional model, allowing competencies to be summarized into a single overall rating score. The reliability coefficient was 0.86, based on McDonald’s omega, and 0.85, based on Cronbach’s alpha.

Regarding external validity, we utilized logistic regression to analyze the relationship between the assessed competencies and being a member of the Group of Talent. Table 4 displays the results of the binary logistic regression with the eight competencies. The analysis indicated that of all the competencies, execution (B = 1.28. p = 0.03) and adaptability (B = 2.00, p = 0.02) were the best determinants of status validation. A second logistic regression showed that the total competency score was the best predictor of the criteria (B = 9.13, SE = 1.68, p < 0.001).

As Table 5 shows, the total competency score accurately predicted that 64 participants would attain Group Talent status and 40 would not. The overall accuracy of these predictions was 87%. Specifically, the model was more effective at correctly identifying those who would achieve Group Talent status, with an accuracy of 89%, compared to an 83.0% accuracy rate for correctly predicting those who would not attain this status. However, the model generated eight false negatives, erroneously predicting that these candidates would not be identified as Group Talent when, in fact, they did achieve this status. Similarly, there were eight false positives, where the model incorrectly identified candidates as potential Group Talent who ultimately did not achieve this designation. We computed the sensitivity analysis by dividing the level of potential and total competency score into high and low and comparing it with Group Talent status. The sensitivity analysis showed a value of 100% as a true positive rate and 15% as a false positive rate.

For research questions 2 and 3 that gauge the fairness and adverse impact related to gender, the first step was to conduct a chi-squared test to ascertain whether the status validation probability differed between the male and female participants of the virtual AC. Table 6 displays that 52 females and 67 males attained status validation as GT. Results did not show a significant interaction between gender and status validation, X² (1) = 0.306, p = 0.403.

Table 8 presents the results of the MANOVA analysis conducted to examine the effects of participants’ gender on various competencies. Table 7 reveals that gender did not have a significant impact on any of the competencies assessed. The overall effect of gender on the combined competencies was not significant, F(16, 222) = 0.614, p = 0.874; Wilks’ lambda = 0.917. This indicates that the participants’ gender did not play a significant role in determining their competencies.

We conducted MANOVA to analyze the interaction between the genders of the assessees and the assessors with respect to competencies. The analysis revealed no significant interaction, F(2, 120) = 1.259, p = 0.226; Wilks’ lambda = 0.834, between the genders of both parties and the competencies assessed (Table 8).

We compared the selection rates for men and women to assess the bias possibility in selecting candidates for the Talent Group, with 62.6% for men and 57.6% for women. An impact ratio analysis resulting in 92% indicated no adverse impact according to the 80% rule, as the selection rate for the less-favored group (women) was not less than 80% of the most-favored group (males), thus suggesting a fair selection process based on the virtual AC too.

Results did not show a significant association between age and status validation, X² (1, 118) = 1.814, p = 0.178.

The MANOVA analysis indicated that age did not significantly impact any of the competencies assessed. As shown in Table 9, the overall effect of age on the competencies was not significant, F(8, 109) = 1.277, p = 0.263; Wilks’ lambda = 0.914. This suggests that the age of the participants did not play a significant role in determining the competencies assessment.

As a final control, we adopted an ANOVA to test if the combination of gender and age of participants could impact the total competency score. The ANOVA results indicated that there were no statistically significant main effects of gender (F(1, 114) = 0.00953, p = 0.922, η² = 0.000) or age (F(1, 114) = 0.05275, p = 0.819, η² = 0.000) on total competency score. The interaction between gender and age also did not reach statistical significance (F(1, 114) = 1.48027, p = 0.226, η² = 0.013). This suggests that neither the gender, participants’ age nor the combination of these two factors had a significant impact on their total competency scores within the analyzed sample.

We conducted the study on a cohort of Italian organizations that had implemented a virtual AC for evaluating 120 employees. The primary aim was to explore the efficacy of the AC in its virtual incarnation, focusing on its validity and fairness as a selection tool. Three fundamental research questions drove the research. First, we sought to determine whether the virtual AC displayed general indications of validity as a selection mechanism. Second, we investigated whether the virtual AC was an appropriate tool for selecting candidates across genders. Lastly, the research aimed to ascertain whether the virtual ACs exhibited any differential impacts on assesses belonging to various age groups.

The findings showed preliminary evidence of general indications of virtual AC’s validity. Regarding internal validity, the findings supported a score based on a unidimensional index and showed good reliability. The results inherent in external validity also showed good and promising validity indications. In summary, the overall rating showed that it could predict the validation process results with a relevant sensitivity. The results we obtained about the validity of the virtual ACs allowed us to align it with conventional ACs (Eurich, Krause, Cigularov, & Thornton, 2009) and the existent literature that confirms the validity of the online selection procedures (Groene, Knorr, Vogel, Hild, & Hampe, 2022).

We also addressed the lack of research on the adverse impact and fairness of digital selection procedures. Results suggest fairness in terms of gender. We found no evidence of adverse impact. This sets up a good robustness signal for digital selection procedures, and virtual ACs specifically, boding well for their use. This finding is in contrast to Lievens, Schollaert, and Keen (2015) research that reported gender subgroup differences when studying digital situational judgment tests in police candidates, which is a predominantly men-dominated profession.

Finally, the participants’ age did not influence the competencies assessment. This result contrasts with a study on a digital assessment performed with computer simulation games that found a better performance for men than for women and younger than for older applicants (Melchers & Basch, 2022). The results may vary because adopting these technologies requires skills equally present across age groups in the roles analyzed. Alternatively, the hypothetical advantages of younger participants – such as familiarity with digital technologies, adaptability to innovations, and technological proficiency – may have been offset by the job experience of older participants. Regardless, nowadays, people may perceive virtual ACs as easy to use and useful for participants, as shown by Pattnaik and Padhi (2021).

The findings have both theoretical and practical implications. The study revealed general indications of validity and fairness for both genders. Through this, we can theoretically imply that a well-designed and valid virtual AC reduces the chances of adverse impact. Researchers have widely established the importance of conventional ACs by demonstrating their validity, reliability, and overall significance (Woehr & Arthur, 2003; Rupp, Thornton, & Gibbons, 2008; Thornton & Gibbons, 2009; Dilchert & Ones, 2009). However, despite living in the industrial era where digitalization has become widespread, literature on virtual ACs is scarce. Thereby, this research contributes to understanding how a virtual assessment works and is an important starting point for future research on AC’s digitalization. The study provides initial validity evidence on a fair virtual AC, because it had no adverse impact when both the participants and the assessors considered gender. The pioneering nature of the research serves as its strength. Furthermore, the present research not only studies the adverse impact of virtual ACs on personnel selection decisions but also adds to the existing evidence of how digitalization can reduce bias, especially in the presence of human assessors (Aichi, Bassiri, Benmokhtar, & Belouaad, 2018).

A key practical benefit of an AC with no adverse impact and fair assessment is the ability to promote equal opportunity by integrating diversity management. The fairness of virtual AC reinforces the need to assess adverse impacts and use inclusive assessment practices. This can translate into how future virtual ACs can consist of a more diverse group of candidates and assessors. Including a pool of assessors from a diverse range of backgrounds ensures a thoroughly comprehensive and potentially fair evaluation process. The composition of a diverse assessor panel is a strong message to the candidates on the organization’s stance on diversity and its commitment to working on equal opportunities.

From the organizational point of view, to gain a more thorough insight, the organization can keep track of the employee’s performance after the AC. This would have reinforcing benefits since the post-assessment data would enable the organization to measure the AC’s efficacy. Although the research shows positive results regarding the fairness of virtual ACs, it is important to pay close attention to these issues consistently. When training assessors for virtual ACs, we must emphasize several key aspects to ensure fairness and accuracy. Assessors should receive training in minimizing potential biases, particularly those arising from video-based assessments, as previous research (e.g. Van Iddekinge, Raymark, Roth, & Payne, 2006; Sears, Zhang, Wiesner, Hackett, & Yuan, 2013) indicates that ratings in digital formats often differ from face-to-face evaluations. Training should also focus on maintaining consistency in applying evaluation criteria across different platforms to ensure the validity of virtual ACs. Moreover, assessors should develop skills to interpret subtle non-verbal cues, which may be less apparent in digital interactions, and mitigate any potential adverse impacts related to demographic factors such as age and gender.

Readers should view the study and its findings while remembering its limitations. The primary limitation of the present research is the lack of generalizability because we based it on a small sample size derived from a single organizational case. To overcome the problem of lack of generalizability, future research should have access to the data of multiple virtual ACs and a larger sample size.

The second limitation was the absence of a job performance variable. Since we based this research on an organizational case, we deemed a significant portion of the data unusable to protect the company’s confidentiality. With the participants’ performance data, this research would have claimed stronger validity evidence (e.g. criterion validity). Future research should include variables that measure performance after the administration of the virtual AC to completely gauge the methodology’s efficacy.

As highlighted in previous research, the third limitation is the need for a more structured comparison between the validity of virtual AC ratings and traditional face-to-face ACs. Studies, including Sears, Zhang, Wiesner, Hackett, and Yuan (2013) and Van Iddekinge, Raymark, Roth, and Payne (2006), suggest that ratings for digital interviews, particularly video-based ones, are slightly lower than those for face-to-face interactions but still maintain validity.

The fourth limitation of this study lies in the criteria for inclusion in the Group Talent, which was not entirely independent of the scores obtained from the virtual AC. However, these scores constitute only one-third of the evaluation sources. Furthermore, we divided decisions regarding Group Talent status from a comprehensive process encompassing various phases involving multiple roles of the human resource department and the line.

Fifth, our study analyzed the validity of the virtual AC by focusing on the total competency score and overall evaluation. The current scientific debate, as discussed by Dewberry (2024) and Bosneag and Iliescu (2024), revolves around whether ACs should rely on dimension-based or exercise-based indicators. This opens a potential area for further exploration in virtual ACs, where scholars could assess the efficacy of both approaches in digital environments to ensure robust and fair selection practices.

Future research into applicant reactions could provide valuable insights into virtual AC. These reactions could be positive or negative, influenced by various factors. Scholars have found digital assessments to increasingly appeal to candidates (Folger, Brosi, Stumpf-Wollersheim, & Welpe, 2021). However, others see them as lacking in procedural justice compared to traditional methods in certain situations (Balcerak & Woźniak, 2021). Moreover, there is a degree of skepticism about them from recruiters’ perspectives (Basch & Melchers, 2021). This underscores the importance of considering applicant reactions in evaluating the usefulness and relevance of assessments.

This complex landscape calls for a multi-faceted exploration of how people perceive digital assessments and how these perceptions influence the selection process’s fairness and efficacy. Importantly, integrating the findings of Baczynska, Skoczeń, Thornton, and Chen (2024) on age’s moderating effects offers a valuable lens through which scholars may examine the interaction between individual differences and digital assessment outcomes. This approach broadens our understanding of digital ACs’ utility and applicability and enriches the discourse on personalizing assessment methodologies to accommodate diverse applicant profiles.

Conventional ACs are valuable and effective but scholars have associated them with practical and logistical problems and biased judgments (Lievens & Thornton, 2017), which research could address by digitalizing them and conducting them using virtual ACs. Our research presents preliminary evidence of validity and fairness for online ACs. However, it is important for future studies to (1) explore multiple virtual ACs, (2) compare virtual and conventional ACs to gauge their effectiveness, and (3) find ways to assess fairness in more subgroups. The research evidence contributes to the knowledge of practitioners and researchers, and the articles serves as a stepping stone for further exploration and understanding. Our results affect the policy field because virtual ACs promote diversity and inclusion based on gender and age. Moreover, virtual ACs can eliminate geographic and physical barriers and thus enable organizations to attract more diverse candidates. Hence, these findings align with broader discussions on the legal and regulatory environment for employee selection across different jurisdictions, which emphasize the need for fair and inclusive selection methods (Sackett et al., 2010). Finally, the implications regard the economic field, too, because virtual ACs can offer significant cost savings by reducing expenses associated with travel, venue rental, and logistical arrangements.