Human coaches and AI coaching agents: an exploratory quasi-experimental study of workplace client attitudes

AI is commonly defined as “a system’s ability to interpret external data correctly, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation” (Kaplain and Haenlein, 2019, p. 15). Developments in AI have profound implications for professional services, ranging from consulting to training and coaching (Passmore and Tee, 2024). The emergence of Generative AI marked a pivotal step in the development of AI, whose use has grown rapidly over the period 2022–2025. While traditional script-based AI operates based on predetermined rules, Generative AI can learn from data and autonomously generate unique content. The coming decade (2030s) is likely to see radical changes in the way coaching and professional services are provided, with the potential of AI solutions to increase accessibility, reduce costs and improve work quality, although the exact pattern is at present challenging to predict (Dell'Acqua et al., 2023).

The impact of AI on coaching is a topic of considerable debate, with divergent views on how an AI coaching agent will integrate into the coaching ecosystem. Three distinct perspectives can be identified. The first is that AI technologies will replace human coaches. A second school of thought is that AI tools cannot replace human coaches due to coaching being contingent upon unreplicable and intrinsic human qualities, meaning that AI is, in effect, unable to coach. Thirdly, AI will complement human coaching in a hybrid model, but design and deployment require careful management to protect individuals and organisations. We have labelled these three perspectives as “Zoomers”, “Doomers” and “Bloomers”.

Examining the first perspective, the “Zoomers”, these proponents argue that AI is a learning technology which, over time, will improve and may ultimately replace human coaches. This perspective is grounded in the rapid advancements of AI technologies (Balasubramanian, 2023) and evidence from AI coaching (Terblanche et al., 2022a). A second school of thought comes from those more critical of AI. “Doomers” argue that AI conversation tools will never replace human coaches due to the unique qualities of being human, which are argued to be essential features of coaching (Bachkirova and Kemp, 2024). Human coaches bring interpersonal skills that facilitate deeper conversations, which AI is unable to replicate and, without which, no worthwhile coaching outcomes can be produced. Lastly, a more integrative view, “Bloomers”, argues that AI has the potential to become a useful part of the coaching ecosystem, but care needs to be taken in managing ethical and practical aspects of its deployment (Passmore and Tee, 2024). However, to achieve a successful outcome, careful management is required in the design and deployment of AI technologies in coaching.

The past decade has provided significant evidence of the potential of human coaches to generate positive changes in clients’ behaviours, cognition and emotions (Wang et al., 2022) and to achieve these outcomes within professional workplace environments (Athanasopoulou and Dopson, 2018). However, research on human coaching comparatively remains sparse compared with other disciplines. Reflecting the complexity of researching an aspect of development which features a personal, confidential relationship, where power in the relationship rests not with the therapist or service provider, but with the consumer or service purchaser and where often a third party, the organisation, is also cautious about its data and reputation.

A systematic literature review (SR) of AI coaching research suggests that the evidence on the impact is scant (Passmore et al., 2025b). The review identified 16 peer-reviewed journal papers (n = 2,312) which met its inclusion criteria and from which these four themes emerged: the potential of AI coaching agents, their current known impact, the risks and opportunities for the coaching industry and coaching ethics. On the theme of potential, the SR noted that AI coaches can already provide feedback, help with goal-tracking, serve as accountability partners and are seen as accessible, convenient and psychologically safe Terblanche et al., (2023, 2024). On the second theme, AI coaching impact, evaluation research has been limited in number and scope, with serious limitations. Studies have typically used student samples as opposed to workplace coaching clients, often focused on health outcomes or health care environments.

Reviewing this evidence, in a study by Ellis-Brush (2021), the participants were coached by an AI coaching agent trained in psychological techniques (specifically, cognitive behavioural coaching) and showed a growth in self-resilience. Hassoon et al. (2021) suggested that cancer survivors coached by an AI coaching agent were able to increase their step count more than those assigned to a control group. Terblanche et al. (2022b) reviewed levels of goal attainment in a comparative study on the effectiveness of AI coaching and human coaching, with a quasi-experimental design featuring data from two separate longitudinal studies, different samples and different time periods. In one study, data were collected using human coaches by one group of researchers in 2018–19. In the second study, AI data were collected from a different sample during the UK COVID-19 Lockdown in 2020. While the AI coach participants achieved similar levels of goal attainment as those working with a human coach, the unique circumstances of the study make generalising the results problematic. Finally, in a study by Bamber (2025) comparing client ratings of a concealed human coach, pretending to be an AI coaching tool and a human coach (a Wizard of Oz study) and claimed to show similar levels of working alliance. Alongside these client-based evaluation studies, one study (Passmore et al., 2025c) used International Coaching Federation (ICF) assessors to evaluate AI bots using the ICF’s coaching competency framework. The outcome suggests that the AI tool evaluated was capable of meeting the ICF Associate Certified Coach (ACC) standards.

Overall, these studies suggest AI’s ability to effectively fulfil many competencies determined as necessary for effective coaching, but, given the limitations of previous studies, it is not yet possible to conclude if AI coaching agents offer comparable performance to human coaches, specifically in the context of real-world organisational coaching. Apart from a focus on goal attainment, few studies have sought to measure the wider features of AI coaching. Such measures might include the development of new client insights, perceptions about the confidentiality of the conversation (Passmore and von Bartheld, 2024), the working alliance (Graβmann et al., 2020), or trust (Schiemann et al., 2019).

Client and extratherapeutic factors are theorised to be the single greatest predictors of outcome variance (Asay and Lambert, 1999) in “helping-by-talking” interventions, with one client trait – their level of commitment–important both to the process and to the completion of subsequent actions at the close of the session (Passmore and Whybrow, 2019). High levels of commitment are associated with increased effort towards both the coaching process and goal attainment, which in turn are connected to a greater likelihood of the client achieving their desired outcomes. Tee et al.’s (2022) systemic literature review highlighted the importance of client commitment, which is reflected as a key theme across multiple studies (Audet and Couteret, 2012; Baron and Morin, 2009; Bouwer and van Egmond, 2012).

The working alliance, or coach-client relationship, has been suggested as another key predictor of coaching outcomes. Graβmann et al., (2020) study results suggested that the working alliance was positively related to desirable coaching outcomes (range: r = 0.32 to 0.64), while negatively related to unintended negative effects of coaching (r = −0.29). However, not all writers have shared this perspective. DeRubeis et al. (2005) suggest that the relationship is mainly determined by the client’s ability to form such an alliance, with those same clients being the ones who make the most progress in therapy and, by extension, in coaching. Similarly, de Haan et al. (2020) evidenced that the working alliance was not strongly related to coaching’s effectiveness. In considering AI coaching, Terblanche and Cilliers (2020) argued that, rather than focussing on working alliances, AI coaching studies should investigate technology acceptance. We assert that, for AI coaching clients, higher levels of technological competence and confidence might contribute towards enhanced outcomes.

A third factor is the clients’ perception of confidentiality, fostering a safe environment for clients to openly discuss personal and professional challenges, which is mentioned repeatedly in coaching practice texts (such as Brennan and Wildflower, 2018; Passmore and Sinclair, 2023; Smith and Arnold, 2023) as well as in professional standards (ICF, 2019). Confidentiality is an important ingredient enabling the development of a trusting relationship and thus promoting a genuine dialogue, facilitating deeper insights and meaningful progress (Ebrahimi, 2024). The issue of trust is associated with confidentiality. Trust is argued to be a critical factor in influencing coaching effectiveness (Graβmann et al., 2020). Boszoremyi-Nagy (2013) has highlighted the importance of trust within clinical relationships as a key building block of the relationship. Clients are more likely to disclose more truthful information when they trust the coach to honour their commitments, from respecting confidentiality to working competently and supporting them towards their goals. Related to trust is shame-embarrassment and our willingness to disclose possibly embarrassing information to others. Ellis-Brush (2021) has suggested that a technology-based conversation can be perceived as a safe space because there is no judgement from other people. This aligns with the online disinhibition effect (Suler, 2004), which suggests that people feel less inhibited in cyberspace than in the real world, leading them to behave differently (Suler, 2016). Coaching evaluation remains a widely debated issue within the field (see, for example, Peterson, 1996; Seay and Muscarella, 2024). While the number of evaluation studies remains small, most have drawn on either psychological tools, such as growth mindset and psychological safety (Passmore et al., 2025a), which were not specifically designed for coaching or organisational use. Given organisational measures are space, researchers must develop and experiment with new and different measures.

Based upon the above theory and research, ten hypotheses relating to the comparison of human and AI coaching (Table 1) were proposed. This is a comparatively large number of hypotheses, but given the lack of research and the breadth of factors within the custom coaching questionnaire (Seiler, 2021), we felt this number was justified. The hypotheses were based on six themes. Participants would evaluate the AI coaches more highly on new insights and goal attainment, but poorly on commitment, working alliance and trust compared to human coaches. Finally, there would be no difference in ratings for AI and human coaches on confidentiality.

The study employed a quasi-experimental, between-participant design. There was one categorical independent variable featuring two non-randomly assigned conditions: human coaching and AI coaching, drawn from different organisations. There were six continuous dependent variables: new insights, positive working alliances, goal attainment, commitment, shame and confidentiality.

A purposive sampling strategy of volunteer participants, who were aged 18 or over, in professional, knowledge-based (white collar) work and wanting to engage in coaching was adopted. Human coaches were drawn from a range of European Union (EU) and recruited directly by human coaches who volunteered to participate in the study. Those in the AI coaching condition were members of a single EU-based organisation and were recruited by the head of learning and development.

The study involved n = 63 participants, of which n = 28 with a mean age of 41 were assigned to the AI coaching condition and n = 35 with a mean age of 47 were assigned to the human coaching condition. Data about the participants’ race and gender are in Table 2.

A total of n = 28 human coaches were recruited through the researchers’ professional networks to participate in the study. Human coaches were ICF- or EMCC-accredited and undertook coaching as a professional activity with clients. Each human coach was restricted to a maximum of three clients. They were required to deliver the coaching sessions online, with each session required to last between 30 and 45 min. There was no guidance provided as to the model or approach that human coaches should adopt.

For the AI coaching condition, this study used “Alpina”, a commercially available AI coaching agent developed by the company “evoach”, with permission for its use obtained from the designers. Alpina was running on a GDPR-compliant platform and used a text-based method of communication using Open AI’s GPT-4 turbo Large Language Model, with additional prompt coding by the developers to enhance Alpina’s coaching capabilities. Alpina was prompted to act as a professional business coach using clean language principles and being supportive as well as empathetic in its communication. It was framed as being well educated in a transformative coaching approach (without specifying which) and adhering to the principles of positive psychology. The AI prompts also included clear instructions about not providing advice or suggestions and how to apply the ICF’s coaching competencies. In line with ICF requirements, a contracting process was built into the AI coaching condition. Table 3 contains an example of prompts provided to the AI coaching condition participants. All participant data from the AI coaching sessions were supplied to the researchers in anonymised form.

A customised measure was used, based upon the EXCBOS coaching evaluation questionnaire (Seiler, 2021), an exploratory instrument for evaluating human coaching. To encourage completion rates, an abbreviated version was created featuring 11 items, each scored using a percentile range and selected from the original 35-item questionnaire. Given Seiler’s original focus, small changes in language were needed to ensure retained items made sense for this study’s AI coaching condition, such as adding the term “AI” to the item. The featured themes were: (1) new insights, (2) goal attainment, (3) commitment, (4) working alliance, (5) confidentiality and (6) shame. An example item is: “I felt I could trust my coach with confidential information”. A positive relationship was operationalised as a mean score of 50% or higher for the theme. H₁₀ used the mean score from the shame items to determine the participant’s level of trust. A list of the questions is provided in Appendix 1. In addition, biodata items relating to age, gender and race were also collected.

For the AI coaching condition, written consent was obtained from the host organisation. An e-mailed anonymous link then invited potential participants to find out more about the research and provide their individual consent to participate. For the human coaching condition, informed consent was obtained from individual participants through a written agreement that described the research.

To ensure comparability, clients in both conditions were offered a single coaching session, after which they were invited to complete the coaching client attitude measure. Data were collected using SurveyMonkey, accessible via the participant’s smartphone or computer. All responses were anonymised. No data were collected from the human coaching condition participants detailing which human coach provided which session.

Ethical approval was secured from a university ethics committee.

Prior to analysis, skewness and kurtosis were assessed for all variables. The Shapiro–Wilk test indicated that none of the variables were normally distributed (all p > 0.05). In addition, two variables (shame and confidentiality) displayed suboptimal skewness and kurtosis. No data transformations were applied due to the small sample size. The means and standard deviations for all variables are displayed in Table 4.

Given the sample sizes of both the AI and human coach conditions, a series of independent-samples Mann–Whitney U tests were conducted.

Participants evaluated their human coaches significantly higher on new insights (Mdn = 82.5) and goal attainment (Mdn = 90) compared to participants who had AI coaches (Mdn = 42.5 and Mdn = 27.75, respectively), and these differences were significant (U = 116.5, p < 0.001 and U = 84, p < 0.001, respectively). Therefore, the null hypotheses for H₁ and H₂ cannot be rejected.

The results indicated support for H₃ and H₄, indicating that participants assessed their human coaches significantly higher on commitment (Mdn = 82.5) and on generating a positive working alliance (Mdn = 89.5) compared to participants with AI coaches (Mdn = 50 and Mdn = 35.25, respectively). Both differences were statistically significant (U = 222, p < 0.001 and U = 507, p < 0.001, respectively).

No significant differences were found between human (Mdn = 94) and AI coaches (Mdn = 98) on participant evaluations of trust (U = 963, p = 0.343). Similarly, there was no significant difference between human (Mdn = 97) and AI coaches (Mdn = 97.5) on participant assessments of confidentiality (U = 446.5, p = 0.528). Therefore, the null hypotheses for H₅and H₆ cannot be rejected.

Human coaching was considered more confidential (H₇) and scored as developing a stronger working alliance (H₈), developing commitment (H₉) and on experiencing no shame (H₁₀) (i.e. feeling judged by the human or AI coach), although the difference between the human and the AI coaching agent for H₁₀ was not significant: Human coach (Mdn = 94), AI coach (Mdn = 98), U = 963, p = 0.343).

The results from this study provide a complex picture, neither supporting the view that AI’s coaching created an existential threat to the human coach (Diller et al., 2024), nor the view that AI coaching agents are not coaches (Bachkirova and Kemp, 2024). Instead, a more nuanced perspective is offered, reflecting both the strengths of AI as an emerging technology, which can engage in one-to-one conversations and its weaknesses at the time of the study in 2024, in high trust conversations such as coaching. In particular, it must be considered that AI’s advancements move at a faster rate than the academic research dissemination process, so it would be wrong to draw conclusions as to all AI coaching’s capabilities in 2025 and beyond, based upon findings generated in 2024.

Nonetheless, based on the results from this small-scale quasi-experimental study with professional, white collar workers, it appears that, as a population, professionals have a general preference for human-to-human relationships in coaching based on a one-off session. Participants rated the human coaches more positively in their ability to create a positive working alliance than the AI coach. Similarly, the human coaches were also rated more highly in terms of client commitment and goal attainment.

Evidence from Graβmann et al. (2020) suggests the importance of the working relationship to effective outcomes such as goal attainment, whereas findings from a large-scale longitudinal Randomised Control Trial (RCT) by de Haan et al. (2020) suggest that the working alliance is not strongly related to coaching effectiveness over the full coaching relationship, only at the start. This might therefore have a distorting effect on the evaluation of a single session coaching intervention, as used in this study, with the recognition that this design has low ecological validity. There is evidence that client motivation and commitment are important factors in the coaching process (Rayiu and Baban, 2012).

Other aspects not explored in this study, such as humour and presence, are currently difficult for AI coaching agents to replicate, as is the contextual sensitivity displayed by some human coaches who have lived experience and direct organisation working experience. This may allow them to understand wider organisational dynamics, cultural factors, or the “lived experience” of their clients.

In this sense, AI at the time of this study was behind the subset of human coaches who bring these elements into their work and, through this, enable clients to feel heard, understood and respected for the unique human being they are (Passmore and von Bartheld, 2024). For those human coaches unable to bring these qualities to their work, an AI coach’s ability to engage at a transactional level makes their coaching offers comparable.

We hypothesised that the AI coaching agent would be rated more highly by participants for both confidentiality and shame (embarrassment). These hypotheses were based on Suler’s (2004) Theory of Disinhibition, which suggests that inhibitions are reduced as distance grows, for example, from face-to-face to online and, by extension, from online human-to-human to human-to-AI coaching agent. Based on this theory, we expected AI coaches to score more highly on the two sub-scales of shame (embarrassment) and confidentiality. However, there was no statistically significant difference between participant ratings of AI and human coaches.

The fact that participants rated the human and AI coaches at similar levels may be related to the participants’ awareness that they were taking part in a research study and that human researchers would be accessing the details of their sessions. In a pure AI coaching session, participant ratings of confidentiality might be increased by making the AI coach more clearly articulate the confidential nature of the conversation, affirming that no human will access the data. Future research is needed to test this hypothesis.

We maintain the view that, for some coaching topics where embarrassment or shame is a feature, the AI coach may offer a more appropriate space for the client to disclose and explore this content. This may only occur where the topic constitutes one that would engender a strong emotional response of embarrassment. Such topics are more likely to occur in, for example, counselling on sexual practices, as opposed to coaching about task prioritisation or career development at work. These effects may therefore be less relevant in work-based coaching. Furthermore, these effects may only be seen when clients feel in control of their data; that is, researchers and others do not have access to the contents of the session.

Aspects which we did not measure in this study include coaching client evaluations of the cost comparison between AI and human coaching or convenience (needing to schedule diaries in advance with a human coach or having access whenever required, 24–7, to an AI coach). AI coaching agents offer significant advantages. They are always on, never get tired, or take holidays and are available at a fraction of the cost of a human coach. AI can also be configured to engage in any language and offer both text and speech options for engagement, making them more accessible for different user preferences or requirements.

Our literature-informed hypotheses suggested that the AI coach might perform strongly on some of the six factors we measured, either matching or outperforming the human coach. However, the results indicated lower average ratings across all factors.

Data from this study showed a large standard deviation (SD) for the client responses to AI coaching, suggesting a wider range of feelings from different participants (Figures 1 and 2). This was most marked in goal attainment and new insights, where some participants rated AI coaching at a similar level to human coaching, whilst others rated it significantly worse. This contrasts with the participant response for other factors such as Shame (Figure 3).

These results suggest that, for some clients, the experience of AI coaching met their expectations in terms of insights and goal attainment and was comparable to the ratings given by participants of their experience with human coaches. However, for others, the AI experience was rated as significantly lower than participant ratings of the human coach experience. Extreme caution is needed, given the small sample size and the use of a custom questionnaire employed in this exploratory study. Alternative explanations may include: client factors such as age (younger participants are more open to AI technology than older users): technology confidence (Terblanche and Cilliers, 2020), with more confident technology users being more engaged and giving higher ratings than less confident users: job role (data roles being more engaged with higher ratings than users from roles focussing on interpersonal relationships): or personality preferences (with introversion or neurodiverse characteristics contributing to higher engagement); as well as the items used in the questionnaire. Furthermore, the text-based nature of the AI coaching agent used in this study required text-to-text communication, encouraging the client to crystallise their thinking prior to committing it to text. Thoughts in human coaching can sometimes be less considered and more emergent, which may have influenced these results.

As AI capability accelerates, further research is needed to explore voice-to-voice coachbots, avatar-based coachbots and coachbots that can meaningfully read client micro-expressions and use this data to shape how they coach. Our hypothesis is that the greater the anthropomorphising of the tool, such as through voice-to-voice (with options on different accents and languages), the use of a 2D or 3D photorealistic codec avatar and other humanising features, each will contribute to higher levels of engagement, a stronger client sense of psychological safety and higher levels of client rating.

The intriguing nature of the results in this study brings into question whether the outcomes from previous student-based and goal attainment coaching studies can be directly transferred to workplace coaching. Further research is needed to explore this question. This study also suggests that our relationship with AI coaches may be more complex than initially assumed, and that AI coaching providers may have a different experience than human coaching, which requires different expectations and assumptions, rather than direct comparison with the human coaching experience.

To date, studies exploring the potential of AI coaches have focused on student populations and goal attainment or used human masking as an AI tool (“Wizard of Oz” designs). This study employed a commercially available AI coaching agent with real workplace coaching clients exploring real-world issues. It sought to evaluate their engagement beyond goal attainment, considering factors such as perceptions of trust, confidentiality and insight. The study’s results suggest that while AI coaches, as of 2024, may have been capable of holding coach-like conversations (Passmore et al., 2025c), users, on average, preferred a human experience, with a portion of participants rating the experience as similar to those rating a human coach, and others rating it significantly worse. As technology continues to evolve, with the development of voice-to-voice and avatar-based tools and as user expectations evolve, different outcomes may emerge.

The primary limitation of the study was the introduction of a custom measure. Although the items were a subset selected from the existing Executive Coaching Behaviour Observation Scales (EXBOS) instrument (Seiler, 2021), the study sought to measure six variables using an 11-item scale and did so as a pilot without conducting any factor analysis. This might account for the high SD values obtained in the dataset. Thus, care must be taken in generalising these findings. A second limitation was the diversity of the samples. While both participant groups were white-collar professionals, the individuals were drawn from different organisations. A range of factors, not least organisational culture, may have impacted the results. Thirdly, individuals were volunteers who chose to engage with either a human or AI coach, as opposed to being randomly allocated and instructed to participate. Factors such as user perception may have impacted the results. Given the relative novelty of AI coaching, those who were offered the AI coach may have seen this as an opportunity to test out this new technology, rather than to engage in meaningful coaching. In contrast, those choosing human coaches may have been aware that human coaching is a well-established organisational intervention. Fourth, the AI coaching condition participants were advised that they could either delete or save the conversation. If they agree to save it, it would then be reviewed by the researchers. Thus, participants were aware that humans would also be viewing their content in detail. This may have impacted on views about the confidentiality of the conversations and shame. A fifth limitation was the relatively small sample size, which allowed for comparative analysis between the AI and human coaching but did not allow for exploration of differences in age, gender, race, or differentiating factors. A further limitation was that the evaluation was based on a one-off (single), coaching session. Attitudes may change if the assessment takes place after, say, the 4th or 6th session. In addition, there was no guidance or requirement for the human coaches to adopt any given approach, structure, or tool, meaning there would have been some variation in the coaching experience for participants in the human coaching condition. Some human coaches may have slipped into “tell” mode, some may have rigidly adhered to prescribed coaching frameworks, some may have detoured into orthodox counselling territory and so on, with this variance making it difficult to treat the human coaching intervention as a meaningful construct for research purposes. Finally, the sample size in this small-scale study was insufficient to detect anything other than large effects. Future replication studies would be strengthened by featuring larger, more diverse samples.

The results from this study challenge previous research findings, which have claimed similar outcomes from engagement with AI and human coaches (Hassoon et al., 2021; Terblanche et al., 2022a). We suggest that greater caution is needed in claiming the comparability of AI coaches with human coaches in applied settings and that more research is needed to better understand client responses to AI coaching. However, given the significant cost and convenience advantage AI coaching presents compared to human coaching, they are likely to have a future role in workplace learning and development. But in designing these agents, the desired outcomes and the user experience should be central, with the aim of creating an easy-to-use and flexible tool that generates individual and organisational benefits.

The research was approved by Henley Business School–Approval number – SREC-HBS-20240209-JOPA9549.

The supplementary material for this article can be found online.