Transparency, reproducibility, and replicability in human resource management research

It is important to clarify the distinct concepts of (1) transparency, (2) reproducibility, and (3) replicability, which have garnered growing emphasis in management research, including HRM (Bainbridge et al., 2017; Rynes and Bartunek, 2017). Transparency refers to research processes’ clear documentation and openness (Aguinis, 2025). Reproducibility is achieving consistent results using the same data and methods (Cortina et al., 2023; Köhler and Cortina, 2021, 2023). Replicability involves obtaining similar findings using different methods or data (Köhler and Cortina, 2021, 2023).

In addition, we need to consider three contextual issues that affect transparency, reproducibility, and replicability. First, the absence of a shared HRM ontology makes addressing these issues challenging. Specifically, multiple definitions of concepts and constructs muddy the waters, and competing theories do not always offer helpful guidance for establishing transparency, reproducibility, and replicability. Pursuing transparency, reproducibility, and replicability is challenging without a unified ontology. Second, a lack of a shared culture of science also results in lower levels of transparency, reproducibility, and replicability (Aguinis et al., 2020). For example, there is variability in how editors evaluate the quality of reviewer assessments and recommendations. Some journals have formal systems for doing so, but others do not evaluate reviewer performance at all.

Our goal is to highlight the importance of establishing the credibility of our findings. But, it is only one side of the coin. We also need our research to be valuable and impact multiple stakeholders and beneficiaries (e.g. students, other researchers, organizations, and society), as discussed in detail elsewhere (e.g. Aguinis and Gibson, 2025).

Figure 1 visually represents our conceptual framework, the TRRUST Model. It includes the following six components: Transparency, Replicability, Reproducibility, Unified Ontology (i.e. shared ontology), Shared Culture of Science, and Trust and Values (i.e. shared values and impact importance). In a nutshell, transparency, reproducibility, and replicability are affected by a shared ontology, science culture (i.e. research culture and scientific values), and values and beliefs about the impact and importance of research.

In the remainder of our article, we address the critical importance of addressing transparency, reproducibility, and replicability to enhance the credibility of HRM research and, more broadly, social and behavioral science research. Moreover, we offer practical guidance on enhancing transparency, reproducibility, and replicability to produce meaningful and impactful research. We begin by focusing on transparency, which forms the foundation for reproducibility and replicability.

Transparency encompasses multiple facets of the research process, including theory development, research design, measurement, analysis, and reporting of results (Boselie et al., 2009; Guest, 2011; Wall and Wood, 2005), which allows other researchers to verify findings and build upon the reported results. The imperative for transparency in research mirrors the importance of transparency in HRM practices. For example, just as transparency in organizational communication enhances employee trust, transparent research reporting similarly enhances trust in scholarly work (Klimchak et al., 2020).

Transparency in how a study was conducted diminishes knowledge gaps between researchers and their audiences (e.g. other researchers, practitioners, policymakers), promoting confidence regarding the execution and interpretation of research. Transparency is needed throughout the research process, including theory development, research design, measurement, analysis, and results reporting (Aguinis, 2025). Each of these stages presents unique opportunities for improving transparency. For example, regarding theoretical transparency, researchers should clearly define their constructs and specify the level of analysis at which they operate (e.g. individual, dyad, team, organization) (Aguinis et al., 2018; Murphy and Aguinis, 2022; Margherita, 2022). Additionally, researchers should construct their hypotheses by reviewing theoretical foundations, identifying key mechanisms and relationships in the chosen theory, and verifying that each hypothesis directly connects to theoretical principles through clear reasoning (e.g. Guest et al., 2021; Kehoe and Wright, 2013; McAbee et al., 2017).

Table 1 summarizes key recommendations for improving transparency across the theory development, research design, measurement, analysis, and reporting of results stages of the empirical research process. In addition to the recommendations, Table 1 includes additional sources that readers can consult to obtain more detailed information and learn how the recommendations have been implemented in current and past research. This demonstrates that our recommendations are actionable, not just “wishful thinking.” Next, we discuss a few illustrative recommendations from Table 1.

Pre-registration and registered reports are valuable tools for transparently outlining research design. Pre-registration involves publicly documenting research questions, hypotheses, methods, and analysis plans before data collection or analysis (Nosek et al., 2018). The Open Science Framework (OSF; https://osf.io) serves as a comprehensive registry for pre-registrations, while other platforms like AsPredicted.org (https://aspredicted.org) offer pre-registration tools but do not function as true registries since users can keep information private and selectively report pre-registrations (Nosek et al., 2018). True pre-registration requires public accessibility to prevent selective reporting. This practice enhances transparency, minimizes researchers’ “degrees of freedom” (i.e. choices available to researchers in collecting, analyzing, and interpreting data, which sometimes introduce bias) and mitigates biases that may compromise reproducibility (Briker and Gerpott, 2024).

However, pre-registration is most effective when combined with other transparency-enhancing practices and is not necessarily tied to the publication process (van den Akker et al., 2024). Registered reports extend pre-registration by integrating peer review into the study design phase. Researchers submit their study protocol to a journal, which evaluates and provisionally accepts the report based on methodological rigor rather than study outcomes (Gerpott et al., 2024). This two-stage process continues with a second review after study completion to verify adherence to the approved protocol and appropriate interpretation of results, regardless of whether findings support the original hypotheses. This approach helps reduce publication bias and promotes rigorous, transparent methods essential for reproducibility (Liu et al., 2025; Gerpott et al., 2024). Moreover, by addressing reviewer feedback early, researchers improve the study’s credibility (Logg and Dorison, 2021).

While pre-registration and registered reports promote transparency in planned research, addressing transparency in developing and reporting hypothesis tests is also beneficial. This is particularly important when considering the distinction between a priori and post hoc hypotheses. Traditionally, many researchers have engaged in HARKing (Hypothesizing After Results are Known), a practice where hypotheses are retroactively created to fit the results. Two types of HARKing are especially detrimental regarding transparency (Murphy and Aguinis, 2019). First, cherry-picking involves searching through data involving alternative measures or samples to find the results that offer the strongest possible support for a particular hypothesis or research question. Second, question trolling involves searching through data involving several different constructs, measures of those constructs, interventions, or relationships to find seemingly notable results worth writing about.

We emphasize that reporting unanticipated discoveries is not detrimental to research credibility. Specifically, unlike HARKing, THARKing (i.e. Transparently HARKing) involves openly acknowledging when hypotheses have been developed inductively based on observed results. This practice can be a valuable investigative technique, potentially leading to interesting discoveries and new theoretical insights. However, it is also important that researchers clearly distinguish between a priori hypotheses and those developed through THARKing to maintain transparency (Hollenbeck and Wright, 2017; Murphy and Aguinis, 2019; McAbee et al., 2017).

Transparent reporting in social and behavioral science research, particularly HRM research, hinges on adopting clearer reporting standards. Professional organizations have made efforts to establish such standards. For instance, the American Psychological Association (APA) has delineated specific reporting requirements for commonly used statistical techniques such as multiple regression and ANOVA. While APA has made strides in this direction, some areas still lack comprehensive guidance. For example, although the APA Publication Manual provides a sample table for reporting multilevel modeling results, it does not explain or justify the included information. Moreover, the manual does not explicitly address multilevel modeling. This leaves researchers without a thorough understanding of why certain elements should be reported, potentially limiting the template’s effectiveness in ensuring comprehensive and transparent reporting of multilevel analyses, which have become common in HRM research.

As part of transparent reporting, presenting the exact p-value is crucial. Without a precise p-value, readers are unable to evaluate statistical significance at varying a priori Type I error rates (i.e. α levels), such as 0.05 versus 0.06 (Aguinis, 2025). Unfortunately, many studies tend to simplify p-value reporting using conventional and arbitrary thresholds such as 0.01 and 0.05. So, researchers should provide actual p-values instead of whether a value has reached a particular threshold (e.g. p < 0.01, p < 0.05).

Related to transparency regarding statistical significance is the issue of transparency in effect sizes, including discussing their meaning for theory and practice. While a p-value informs research consumers on whether an effect or relationship is unlikely to be different from zero in the population, it is not informative about the size of the effect. APA has advocated the inclusion of effect sizes in research reports since the publication of Cohen’s (1994) article (Wilkinson and Fowler, 1999). Reporting effect sizes such as correlation coefficients (e.g. Pearson’s r), regression coefficients (e.g. b), the proportion of variance explained in outcome variables (e.g. R²), and standardized differences between group means (d) go beyond simply stating numerical values. For instance, when reporting an R² = 0.20, researchers should interpret the meaning of this proportion of variance explained. Is this effect theoretically important? Is it practically meaningful? Researchers contribute to refining and developing HRM theories by reporting and interpreting effect sizes. Also, in terms of practice, effect sizes guide HRM professionals in deciding which interventions or practices might have the most significant impact. To enhance the practical interpretation of effect sizes, researchers can use contextualized benchmarks (Bosco et al., 2015) relevant to their field of study rather than relying solely on general or outdated guidelines (i.e. Cohen’s “small,” “medium,” and “large” effects). They can contextualize effect sizes by comparing them to those found in similar studies or meta-analyses (e.g. Ng et al., 2024). Researchers should also consider using multiple methods to illustrate the practical significance of their findings (Bosco et al., 2015). This could involve translating effect sizes into more tangible metrics (e.g. dollar values, time saved, or productivity increases) or using visual aids to demonstrate the meaning of the magnitude of effects.

Generative artificial intelligence (AI) tools, such as ChatGPT and other large language models, pose additional challenges regarding transparency (Delios et al., 2024; Gatrell et al., 2024). To address these concerns, it is important to document how AI tools were used and to verify AI-generated content (Budhwar et al., 2024; Delios et al., 2024). More generally, practical tools are being developed to assist researchers in responding to the growing need for transparency in research. For example, the Research Transparency Index (RTI) is an automated assessment tool designed to improve transparency in manuscripts (Aguinis et al., 2024a, b; available at https://www.hermanaguinis.com/RTI.html). Developing and implementing transparency-enhancing practices and tools represents significant steps toward more rigorous and trustworthy research practices. As these approaches evolve and adapt to changing transparency norms, they can significantly and positively impact how research is conducted and reported across various disciplines, including HRM.

As mentioned earlier, reproducibility refers to the ability of different researchers to arrive at the same results when using the same dataset as in the precedent study (Bergh et al., 2017; Cortina et al., 2023). Thus, it is important for verifying claims, supporting evidence-based conclusions, and building cumulative knowledge. There are two types of reproducibility studies. Literal reproducibility refers to a different researcher using a “precise repetition of the analysis of the original researcher” (Cortina et al., 2023, p. 172). In contrast, constructive reproducibility relies on the same data but applies a superior analytic approach to make findings more robust (Cortina et al., 2023). Trustworthy cumulative knowledge through reproducibility is crucial for advancing the field and should motivate researchers to contribute to HRM research (Aguinis and Solarino, 2019). Table 2 summarizes recommendations for improving reproducibility and sources to obtain more detailed information and learn how the recommendations have been implemented in past research. Although Table 2 offers detailed information, we discuss an illustrative subset of the reproducibility recommendations below.

One of the most critical steps in improving credibility is fostering a culture of data sharing (Cortina et al., 2023). As key stakeholders in the research production process, researchers, journal editors, professional organizations, and funding bodies have a significant role in implementing this recommendation (Aguinis et al., 2020). By adopting and enforcing policies that encourage researchers to share their data openly or make them available upon request, we can enhance the reproducibility of research. Furthermore, data-sharing platforms, such as institutional repositories and third-party services like the Open Science Framework (i.e. https://osf.io/), provide secure environments for storing and sharing data (Schwab et al., 2023), as well as protective options that respect privacy and confidentiality requirements (Center for Open Science, 2018, 2023).

Understandably, there may be a reluctance to share data, often stemming from concerns about confidentiality when sensitive information about employees, organizations, and performance metrics is involved or in situations where such data are protected by non-disclosure agreements (Tedersoo et al., 2021; Watson, 2022). Therefore, developing and standardizing anonymization techniques and proper data handling protocols are crucial in protecting participants’ privacy while allowing other researchers to potentially reproduce results (DeCelles et al., 2021). Providing clear guidelines for anonymization and ethical data-sharing practices, as noted in Table 2, would alleviate many confidentiality and ethical concerns.

Encouraging independent reanalysis of datasets is another recommendation that can significantly enhance reproducibility (Köhler and Cortina, 2021). This practice helps identify potential errors or oversights in the original analysis and provides additional validation for the findings (Cortina et al., 2023; Schwab et al., 2023). Journals could create special sections dedicated to reproduction and reanalysis, giving researchers the space and incentive to engage in this important work. Additionally, incorporating independent reanalysis into peer review processes, where reviewers or designated analysts are tasked with reproducing or replicating key analyses before publication, would further strengthen the reliability of published findings. Clearly, the current peer review process model would need to be revised, given that many journal editors and reviewers are already overburdened.

Conducting analyses with and without various statistical controls enhances reproducibility by testing the robustness of results (Atinc et al., 2012; Bernerth and Aguinis, 2016). As Cortina et al. (2023) noted, studies that contain large numbers of control variables might point to the fact that “authors sometimes hunt for the permutation of controls that allows them to conclude support for hypotheses” (pp. 183–184). Adding or replacing control variables allows researchers to demonstrate the strength of these variables’ influence and encourage adequate reproducibility by highlighting which control variables were used and in what manner (Atinc et al., 2012). Additionally, by systematically removing certain controls, researchers can identify whether a more parsimonious model will provide interpretable results (York, 2018). Furthermore, researchers promote clarity and make it easier for other scholars to reproduce their work by running simpler and more parsimonious models. By presenting results from both a critical or complex model and a simpler model, researchers paint a fuller picture of the relationships between variables, allowing for greater reproducibility.

Whereas reproducibility studies are those that reanalyze the same data from a precedent article (Bergh et al., 2017), replication studies refer to those that test similar hypotheses with new data (Kraimer et al., 2023). Replication studies may also be categorized along literal and constructive lines: a literal replication faithfully reproduces the original study’s data collection techniques, research methods, and analyses to test the original findings’ reliability and robustness, whereas a constructive replication involves conducting the same study with slight modifications to the original methodology, measures, or sample (Aguinis et al., 2024a, b; Cortina et al., 2023; Köhler and Cortina, 2021). Additionally, generalizability studies, a type of constructive replication, add substantive moderators to existing studies (Köhler and Cortina, 2021), thus clarifying the boundaries of extant theories and applying findings across varied contexts and populations to understand the conditions under which results hold (or do not) (Aguinis et al., 2024a, b). Constructive replications, including generalizability studies, are more likely to make unique and value-added contributions to the literature than literal replications. This is the likely reason a review of 84,834 articles published from 2010 to 2021 in the 100 highest-impact psychology journals reported that only 0.2% (i.e. 169 articles) were literal replications (Clarke et al., 2024). Table 3 presents key recommendations for improving replicability, and we discuss a subset of them next.

Literal replications may be conducted by the same researchers who conducted the original study (i.e. a dependent literal replication) or by those not initially involved (i.e. an independent literal replication). As such, dependent literal replications can address issues such as sampling error in the precedent study, whereas independent literal replications may also address the original researchers’ reporting accuracy or even malfeasance (Köhler and Cortina, 2021). Literal replications increase confidence in findings, identify biases and errors, and build accumulated knowledge, improving credibility in research.

Literal replications additionally offer students opportunities to familiarize themselves with the research process and the decisions the original authors faced when designing, collecting, and analyzing the data in precedent studies (Schwab et al., 2023). Doctoral students experientially learn research design and how to execute empirical research from precedent studies. In conducting literal replications, it is important to choose precedent studies that have made important contributions to the literature and provide detailed information regarding all methodological procedures (Schwab et al., 2023). Also, it is important not to frame replication studies as “hit jobs”: studies that are perceived, whether fairly or not, as being explicitly conducted to discredit or undermine the original research and its authors. Instead, it is essential to emphasize how the replication study is making value-added contributions on its own.

One of the significant challenges of replicability in HRM research is the influence of contextual factors, such as organizational culture, industry type, geographic location, and workforce demographics (Aguinis et al., 2024a, b). A generalizability study may, for example, reconsider the design of an original study to broaden sample locations or introduce new control variables relevant to the research context (Aguinis et al., 2024a, b). Schwab et al. (2023) offered several examples of generalizability studies that test the contextual boundaries of original studies, highlighting replications of the Milgram and Asch social conformity experiments that expanded contextual boundaries to new organizational and geographic domains. These approaches can help researchers understand whether the findings are specific to certain environments or have broader applicability, thereby enhancing credibility.

Once data have been reanalyzed with modified or new procedures, researchers can compare the new results against those from the precedent study, paying close attention to changes in the effect sizes and their statistical, theoretical, and practical significance. Substantial differences may imply limitations or welcome improvements in the original study (Cortina et al., 2023; Hollenbeck et al., 2006). So, researchers should reflect on these differences and the motivation to test them in their reporting (Aguinis et al., 2024a, b). For instance, researchers should explain if they elected to use a more sophisticated method due to the nature of the data (e.g. non-normal distributions that violate assumptions of the general linear model) or if additional variables were included to mitigate bias. Highlighting differences helps assess the original findings’ robustness and demonstrates how choices can impact conclusions, enhancing the credibility of the replication and offering important insight for HRM theory and practice.

Scholarly research is currently experiencing a credibility crisis in HRM and many other fields, with numerous studies exhibiting deficiencies in transparency, reproducibility, and replicability, which, in turn, compromise their credibility (Hardwicke et al., 2022). Crawford et al. (2022) underscored this issue by attempting to replicate 19 prominent empirical studies, only six of which yielded successful replications. Such problems highlight persistent challenges related to research credibility, ultimately undermining scholarly findings’ ability to generate positive organizational and societal improvements (Aguinis, 2025). Based on the TRRUST conceptual framework shown in Figure 1, our article addressed the pivotal roles of transparency, reproducibility, and replicability in strengthening the credibility of research. Furthermore, we offered specific and actionable recommendations for improving transparency, reproducibility, and replicability throughout the research process, including theory development, research design, and data analysis.

We want to emphasize that there is no “silver bullet” solution for addressing the credibility crisis. In other words, no single recommendation listed in Tables 1–3 can fully address the research credibility crisis in isolation. For instance, pre-registration can enhance research transparency, though it is not a complete solution (van den Akker et al., 2024). However, when researchers implement multiple recommendations in combination, they can offer powerful solutions for addressing transparency, reproducibility, and replicability. Research credibility is not a binary variable; instead, it is a matter of degree. So, the larger the number of recommendations implemented for enhancing transparency, reproducibility, and replicability, the better. Notably, we referred to many published articles that support and have implemented our recommendations, underscoring that these suggestions are actionable, practical, and realistic.

We believe the recommendations for improving transparency, reproducibility, and replicability summarized in our tables are valuable for both novice and seasoned researchers. These guidelines empower researchers to adopt more credible research practices, making our article informative and directly applicable to advancing scholarship. Moreover, journal reviewers and editors can refer to these recommendations when evaluating manuscripts for possible publication, and doing so may help minimize the number of future retractions. Like all knowledge produced in the social and behavioral sciences, and HRM in particular, the issue of addressing the credibility crisis is an ongoing process. Given science’s self-corrective nature, we look forward to future research building off our recommendations to continue to help us move forward with our lofty yet attainable goal of producing rigorous, credible, and impactful research that benefits individuals, organizations, and society.