Executive Personality Assessment With Large Language Models: Updating an Existing Tool and Advancing Similar Measures in Strategy and Management Research

This chapter describes the redevelopment of the Open Language Chief Executive Personality Tool (OLCPT), a language-based machine learning (ML) tool for assessing executives' traits along the five factor model (FFM) of personality (openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism). Whereas the initial release of the OLCPT demonstrated the viability of using supervised machine learning to unobtrusively assess executives' personality traits, recent advances in artificial intelligence (AI) related to large language models (LLMs) warranted revisiting its development. After applying LLM embeddings and performing other updates, including expanding the training sample, the redeveloped tool (available at https://zenodo.org/records/10800801) achieved substantially higher convergent validity than the initial release. The updated tool also demonstrates strong discriminant validity and reliability, and it can measure traits not included in the initial version (narcissism and humility). These improvements demonstrate the potential value of continuously updating existing, computer-aided measures in strategy and management research. Yet, such efforts may not always be feasible or even necessary. Thus, we also use this chapter to offer guidelines for determining when updating similar measures is worthwhile, urging scholars to carefully consider how existing tools perform and the relevance of advancements to the technologies underlying them. We conclude with additional suggestions for advancing measurement in our field, including keeping up with emerging technologies, encouraging complementary approaches to enable triangulation, avoiding the use of advanced techniques without carefully considering their applicability in a given context, and being realistic about what we ask for during the review process and what we consider a meaningful contribution in our field.