Activity theory as framework for analysis of workplace learning technologies: the case of generative AI conversational agents

With the changing nature of work in the context of the knowledge economy and facilitated by digital transformation, on-the-job learning is becoming more relevant and necessary than ever before. There is a growing need for learning in the workplace to rapidly upskill and reskill workers, as jobs become increasingly fluid and ever more complex. Learning and development is of renewed interest to policy makers to enhance employability and organisational competitivity (Morisson and Pattinson, 2021), and it has progressively secured a prominent position in organisations, currently topping corporate agendas (Linkedin Learning, 2024). Traditional formal ways of learning are not flexible and targeted enough to respond to this demand. The disconnect between structured learning and informal learning through work has been documented in previous works (Lizier and Reich, 2020; Martinez-Marroquin and Male, 2021). For effective on-the-job learning, the focus needs to shift away from structure and assessment and “towards the interplay of organisational complexity, fluidity of work, and experiences of learning primarily through work” (Lizier and Reich, 2020). This is of special relevance to knowledge-based organisations, which use knowledge as an asset to solve problems and undertake largely distributed work with technology mediating workers’ interactions. Employees evolve into learning workers, and organisations value their ability to learn and adapt more than a set of pre-established knowledge and skills.

In this context, interest is increasing in generative artificial intelligence (GenAI) as a tool with the ability to scale learning. Like other developments in the field of artificial intelligence (AI), GenAI is a general purpose technology that can be used across numerous applications, many still to be determined. The use of GenAI in education has attracted remarkable interest mainly after the mainstream adoption of generative pre-trained transformer (GPT) tools which triggered a plethora of discussions about their use and misuse. GPTs are AI-based natural language processing systems that provide context-specific responses to queries. They can be used in a variety of learning tasks, from answering simple questions to engaging in more complex conversations. The exploration of GPTs for learning draws mainly on use-cases developed for schools and higher education (HE), where learning is largely structured in modules of set duration, prescribed content, unified learning activities and preestablished learning outcomes. The use of GenAI in informal professional and workplace learning is still underresearched and lacks a theoretical framework to inform the design of applications and the analysis of their impact. Other frameworks are focused on ethical aspects of AI, such as Ashok et al. (2022), or support learning design in the workplace rather than the design of the technology that supports workplace learning, for example, Thongprasit and Wannapiroon (2022). To the best of our knowledge, there are no other specific frameworks for the design of technology for workplace learning. Current research on the use of GenAI in informal learning is based on either observations of its use in the workplace (Wilkens, 2020), which is rapidly evolving and hardly representative of its full potential, or on the analysis of the learner experience when GenAI is embedded in structured training modules (Bucher et al., 2024). These retrospective studies are looking at the use of GenAI at a point in time in particular settings without providing a comprehensive view of its potential impact.

This paper addresses the following questions: What are the functional requirements for technology to facilitate workplace learning, and how do they compare to ChatGPT’s affordances? The present study overcomes the limitations of previous research by providing a forward looking approach of the use of GenAI in workplace learning deduced by using activity theory (AT) as a conceptual framework. We chose AT for its extensive use to study pedagogy, for the analysis of learning experiences and to guide the design of information systems, with focus on human computer interactions. The aspect of this theoretical approach that makes it particularly suitable for our study is its special attention to the mediation role of tools facilitating and influencing the learning process.

Most of the learning that occurs in the workplace is informal and social, it is difficult to measure and it is often unrecognised (Littlejohn and Margaryan, 2014). Informal learning is intentional, but it is not institutionalised; it is typically not structured in terms of objectives, time or learning support, and may occur in the workplace, community and daily life (ASCED, 2014). It is different from formal education, which is institutionalised and leads to a recognised qualification, and from non-formal education, defined as any “structured leaning undertaken through an institution that does not lead to a recognised qualification” (ASCED, 2014). The focus of the paper is on informal learning in the workplace. In this paper, we refer to traditional forms of learning as those used in formal and non-formal education. They have proven to be an efficient way to make a large group of learners reach specific learning outcomes in a structured environment. These forms of learning, however, do not support what graduates need once in the workplace, where learning is unstructured, continuous, self-regulated, largely directed by the individual, complex to assess and is generally a response to new challenges at work (Littlejohn and Margaryan, 2014).

Technology enhanced learning has experienced a dramatic growth in recent years, and it is widely used in structured learning settings, including learning management systems, interactive whiteboards and online video technology (Shen and Ho, 2020). The incremental adoption of these tools at work, however, falls short of addressing the distinct requirements of professional learning (Margaryan and Littlejohn, 2014). Within organisations, knowledge management systems are well established as a tool to organise and share knowledge. Across organisations, learning is facilitated by social learning platforms, where online communities interact, bookmark, and share knowledge and resources. Digital platforms also facilitate access to online learning assets, such as MOOCs (Lin and Cranton, 2015), micro-learning sites, virtual workshops or remote conferences (Bonina et al., 2021) and online communities (e.g. Slack or Discord). With the abundance of information comes the need to efficiently navigate learning content; semantic networks allow the visualisation of knowledge assets and their relationship to facilitate integration of information (Ruiz-Calleja et al., 2019). Over time, Learning Management Systems are evolving into Learning Experience Platforms to respond better to individual needs and provide greater flexibility (Abuhassna et al., 2020). In parallel, there is a growing interest in measuring learning in the workplace. Learning analytics have been explored to measure informal learning that occurs on personalised learning environments (Conde et al., 2011; García-Peñalvo et al., 2014) and in social semantic servers (Ruiz-Calleja et al., 2019). Other emerging technologies are finding their way into today’s workplace for immersive learning experiences through virtual and augmented reality (Moghaddam et al., 2021) or to enhance employee’s engagement and performance through gamification (Thomas et al., 2022).

These platforms, however, were not designed with the primary objective of supporting workplace learning (WPL). An integrated approach to technology-enhanced professional learning remains unresolved (Ley, 2020). There is growing interest in assessing the suitability of existing and new technologies as enablers of workers’ learning (O’Donnell, 2022), with recent attention focussed on the potential of intelligent conversational agents, such as ChatGPT (Wilkens, 2020).

We use AT’s theoretical framework to conduct a systematic analysis of on-the-job learning. AT is a well-grounded theory that has been extensively used for the analysis of complex human activities (Hashim and Jones, 2007), where the subject is involved in an activity directed towards an object with a certain desired outcome. The activity is influenced by the instruments and tools used, the community that the subject belongs to, and the kind of collaboration occurring in the community, guided through rules and division of labour. Engeström (2001) visualized these aspects as depicted in Figure 1. AT facilitates an analysis of the activity, including identification of contradictions as accumulated tensions within the activity system, which can be used as drivers for change (Engeström, 2001).

AT has been extensively used for interpretation of qualitative studies that involve observations, interviews, case studies, questionnaires, focus groups or other methods of data collection (Hashim and Jones, 2007). Using this kind of analysis for workplace learning is challenging because learning in the workplace is unstructured, mostly invisible, difficult to quantify and, overall, its underlying process remains largely unknown. As explained by Margaryan and Littlejohn (2014), there is a need to improve the workplace learning research methodology moving away from “simplistic, self-reported, retrospective methods that focus on what people say they do, rather than what they actually do” (p. 178). Furthermore, we argue that there is a need to adopt a prospective approach to envisage how workers could enhance their professional learning beyond what they currently do. AT is particularly useful for this study given that it also considers expansion and transformation as a result of contradictions, which occur when there are two conflicting practices (Engeström, 2001). Using the concept of expansion on the contradictions between traditional and workplace learning, our research goes beyond observation and description (discovery) of current uses and proposes a prescriptive approach (invention) as represented in Figure 2.

We elicit the requirements that facilitate learning in the workplace in a systematic way, conducting a scoping review of research literature about workplace learning through the lens of AT. We model workplace learning as an activity system to expose the aspects that differ from traditional learning and create tensions when technology designed for education is adopted at work. We derive the functional requirements from these tensions and evaluate the affordances of an existing learning technology against these requirements. Without loss of generality, we choose ChatGPT as the learning technology.

The scoping review builds on Brandi and Iannone’s (2020) review and expands to publications between 2019 and 2024 in Google Scholar, which is chosen because of its interdisciplinarity, academic reach and international standing. The screening and assessment were conducted in accordance with the Preferred Reporting items for Systematic Reviews and Meta-analysis (PRISMA) guidelines (Page et al., 2021). The search terms in the articles’ title were “workplace learning” OR “learning at work” OR “work related informal learning” OR “professional learning” OR “professional development” OR “work related learning” OR “self-regulated learning” AND informal learning contained in the title or abstract of the articles. The search originally landed 161 articles and was further refined by NOT “teaching”, NOT “school”, NOT “teacher”, NOT “student”, resulting in 108 papers. Papers were included only if they were focussed on the key aspect of our search, which is knowledge workers’ informal professional learning. Studies with a different scope were excluded, such as those related to blue-collar workers, focused on formal learning activities in the workplace, or focused on the impact of workplace learning on different aspects of the organisation rather than the learning activity itself. We conducted a first screening of papers based on their abstract, which discarded 87 papers, and a second one based on the full text, which retained 13. The 13 papers that met the selection criteria were analysed in terms of the elements of AT (section 5).

We focus on knowledge workers as subjects, who aim at developing knowledge and skills for increased professional relevance and competency (desired outcome). Following Mwanza’s (2001) Eight-Step Model, we identify the elements of the activity system for workplace learning (Table 1).

Engeström (2001) identified the principal sources of tensions in an activity system within elements of the activity (primary contradictions), such as shortcomings of the tools used.

Rather than a traditional thematic synthesis of the selected literature, the selected papers are analysed through the lens of the WPL activity system, that is in terms of the elements in Table 1. We identify the primary contradictions of workplace learning with traditional learning as follows, and provide a summary in Table 2.

In WPL, the subject tends to be a self-motivated, self-regulated and engaged learner, aware of their learning needs and actively searching for knowledge or skills that assist to overcome work challenges (Kittel et al., 2021; Kittel and Seufert, 2023). The workplace learner is keen to embed learning within work rather than setting time aside for separate learning. Different roles in the workplace may relate to different ways of learning, from a more directed approach to a more unsupervised approach in roles where greater autonomy is expected (Santoro, 2022). Workplace learner is not necessarily seeking credit or certification, the motivation is in learning for doing (Taheri et al., 2022). These subjects see web-based, networked and distributed content as the primary means to access knowledge, which is readily available, as opposed to standardised structured blocks of knowledge often with limited relation to immediate needs (Lucena Barbosa and Borges-Andrade, 2022). Tensions arise between the traditional assumption of learners as passive recipients of structured learning activities and workers expectations to embed their learning within work activities in a fluid and as needed manner (Contradiction C01).

Traditional instructional design assumes that the learning outcomes are known, and drive the course development, delivery and assessment design. However, in the workplace, learning occurs in an unstructured way, without a set syllabus, often without explicitly predetermined learning outcomes (Brandi and Iannone, 2020). Tensions emerge with rules for courses, units of study or other administrative structures that frame formal learning and create a disconnect between learning and work (Contradiction C02). Tensions also emerge with rules around on-campus delivery or through standardised activities in the learning management system (LMS), as opposed to personalised learning embedded in the professional activities (Decius et al., 2019) (Contradiction C03). The outcomes of professional learning should be recognised; however, traditional indicators that quantify formal education and training cannot measure informal learning. There is a contradiction with traditional uniform assessment, largely removed from work practice (Contradiction C04).

Formal learning tends to be about acquisition of knowledge, while WPL is about demonstrating ability to apply the knowledge (Brandi and Iannone, 2020; Pradhan and Rattanawiboonsom, 2024). WPL occurs while participating in work practice, collaborating with peers and advisers and addressing new challenges that provide learning opportunities (Gáthy-Stéber, 2022). In traditional learning settings, learning follows a linear approach often disconnected from the work challenge. A tension is identified as professional learning increasingly shifts the focus from attributes of the learner (competencies, knowledge, skills) to the attributes of the work practice (challenges, opportunities, problems) (Contradiction C05). Learning goes beyond knowledge acquisition to become a complex purposeful activity involving social, personal, organisational and cultural aspects.

Professional learning is a means to an end, to conducting professional activity and resolving professional challenges (Brandi and Iannone, 2020; Graßmann and Decius, 2023). This creates a tension with historical forms of learning where learning is the end in itself (Contradiction C06).

WPL involves distributed content and peer interactions within the organisation and across organisations (Moore and Klein, 2020; Lee et al., 2022; Brandi and Iannone, 2020). This presents a tension with traditionally confined and hierarchical communities related to formal education settings (Contradiction C07).

In WPL, the individual sets goals contextualised by work needs, identifies useful resources, and decides on ways to assess success of the learning process (Santoro, 2022). The provision of knowledge shifts from a centralised model to a distributed model where individuals become both learners and providers of knowledge. The teacher/learner activity expands beyond the workplace environment and into the professional learner’s life, where a range of online collaborative tools facilitate ubiquitous communication. In this context, individuals become life-long learners, and workplaces become hubs for project and problem-based learning. The distributed approach to learning at work creates a tension with the centralised control and unidirectional approach in traditional forms of learning, where teachers are the subject matter experts through the studies (Contradiction C08). In WPL, peers tend to be the knowledge providers, often in personalised manner (Lucena Barbosa and Borges-Andrade, 2022). The learner–teacher relationships are dynamic in that each employee can switch roles between teacher and learner at different times and depending on their background, experiences and objectives. Workplace learners largely self-organise and self-regulate their learning, which creates a tension with traditional division of labor teacher-student (Contradiction C09). Informal WPL is linked to human resources management practices (Khandakar and Pangil, 2019) and supervisors and organisational leaders can influence the uptake of WPL (Taheri et al., 2022). Another tension arises with division of labor in relation to assessment, which shifts towards peer assessment, often involving PDRs or other forms of performance appraisal (Contradiction C10).

WPL is unstructured, distributed and social (Brandi and Iannone, 2020). This creates a contradiction with predominant learning technologies, largely directive and centred on individual study (Contradiction C11). For instance, a tension emerges when learning workers can conduct searches and gain knowledge online, rather than attending a classroom, physical or virtual (Karhapää et al., 2024) (Contradiction C12).

Table 2 summarises the contradictions found in Section 5 and presents the requirements that address each contradiction. These requirements are expressed as heuristic statements of what the desired WPLT shall do to resolve the contradition and support an effective WPL. By listing the functions the application shall perform, these requirements provide guidance to conceive, design, implement and operate the technology use-cases.

The functional requirements, found in Section 6, show overarching themes about provision of readily available access (R02, R03, R21); content to facilitate mimetic, personalised, and situated learning (R01, R04, R17, R23); certification of learning based on application of knowledge (R05, R06, R07, R20); rating of quality and relevance of knowledge assets (R13, R14, R15, R18); support peer and automated recommendations within and across organisations (R11, R12, R19, R22); and maintenance of individual and organisational knowledge inventory (R08, R09, R10, R16).

These requirements provide practical guidance for the design of enabling technologies and set a framework for benchmarking and evaluation of existing technology applications. In this section, we illustrate their practical use by comparing them to ChatGPT’s affordances. The result is a reflection on ChatGPT’s suitability for WPL and on the aspects that would need to be addressed to make it more suitable. The interest of the case study is to illustrate the practical use, rather than an assertion of an end state, acknowledging that Generative AI is still evolving fast. We assess ChatGPT, powered by GPT-4, as paradigm of conversational agents (Achiam et al., 2023).

ChatGPT is primarily a language model designed to provide human-like responses to questions, which can be posed by learners in a conversational and interactive way that can be seamlessly intertwined with work (R23). Workers can, at any time, ask ChatGPT a question related to a work challenge, and they will get a prompt response. It operates on a cloud-based infrastructure, which makes it ubiquitous, networked and readily available if there is connectivity (R03). Because it is trained on a large and varied amount of data, it can simulate conversations and support knowledge acquisition, providing continuous learning opportunities (R02), through questions and answers; and due to its semantic models, ChatGPT stores knowledge linked to practical applications (R15) that can be retrieved through queries about its practical use (R16). However, ChatGPT’s performance is conditioned by the GPT-4’s training, and the output depends on whether the topic has been covered in the knowledge dataset. Besides, although it is trained with large knowledge databases, the content is not validated, which is a limitation for the accurate and contextualised learning desirable in the workplace. Furthermore, GenAI can create misleading nonsensical content (hallucinations). Although GPT-4 has increased the likelihood of producing factual responses compared to its predecessors, it is still designed to be creative and generate new content. Further reliability is therefore needed for ChatGPT to be positioned as a learning technology with trustworthy content.

In its current form, it does not allow individual’s contribution to the knowledge dataset, which limits ChatGPT’s ability to facilitate mimetic learning, defined as learning that occurs from peers or mentors (R01). It also limits collaborative interaction where people learn from each other (R17), shifting between learner and teacher roles, and it is not clear how the knowledge base is maintained, including updating content and retiring knowledge assets as they become obsolete (R18). Remarkably, it does not register the knowledge authorship, nor it tracks the demand and use of that knowledge in a way that allows rewarding sharing of knowledge to facilitate peer learning (R07). The ability to rate ChatGPT responses is still limited to providing feedback with the primary purpose of improving the training of the machine learning models that support ChatGPT, rather than to support peers learning in terms of quality (R13) and relevance (R14).

AI has a long history of use for content recommendation, for instance, in applications such as Spotify, YouTube or Netflix, and more recently in LinkedIn Learning. However, ChatGPT itself does not have the ability to provide personalised and contextualised peer recommendations (R19). It can only provide automatic recommendations on learning relevant to work activities (R12), including suggestions of skills necessary for certain jobs (R11), if prompted and the training data covers the relevant information. Furthermore, peer recommendations at different hierarchical levels and across different organisations (R21, R22) are not currently available and would require further development.

Similarly, AI has been used for user profiling and has the potential to learn about the learner as demonstrated in the field of learning analytics. However, without integrating user history or preferences, ChatGPT cannot adapt dynamically to individual learning needs and falls short from providing personalised answers based on the learner’s background and learning pace (R04). In relation to validation of learning, while ChatGPT can facilitate basic assessment of learning by providing learning content, asking questions and engaging in conversations around application of knowledge (R05), it cannot, in its current state, directly assess achievements of learning, recognise individuals’ levels of expertise over time (R06) or facilitate certification of knowledge (R20). Therefore, without the ability to track user progress or to verify application of learning in workplace tasks, its role in assessment is limited. AI can gather data from patterns of questions and answers, and track user interactions with learning content. However, without integration of learning analytics, ChatGPT does not support the creation of a knowledge inventory (R08), at individual (R09) or organisational level (R10).

In summary, ChatGPT meets some of the functional requirements; for others, it does so only when the training dataset covers the specific content; and for others it just falls short from meeting the functional requirements and would need integration with other tools (eg. Khan Academy and LinkedIn Learning) or further development of specific use-cases. Nevertheless, the functional requirements that GenAI conversational agents meet, in generic forms like ChatGPT or extended with domain-specific training, make them more suitable for WPL than technology solutions designed for traditional education. Further work is required to develop effective ways to incorporate domain-specific content, keep it updated and retire it when it becomes obsolete. Training on domain-specific workplace knowledge through customised datasets is necessary to avoid misinformation, which is particularly concerning when high accuracy is required. The veracity of the responses can be addressed with applications that source content from trusted databases or institutional knowledge management systems, which are areas for further research and development. Likewise, ChatGPT would need further development to support situated assessment and certification of application of knowledge, which presents an opportunity for further research. Large language models are being embedded in a growing number of applications, beyond ChatGPT and other conversational agents. The development of these additional applications provides an opportunity to embed the functionality requirements currently missing for enhanced workplace learning, including recommendations and inventory of individual and organisational knowledge.

Beyond the functional requirements, ethical aspects need to be considered. These have been summarised in Hagendorff’s (2024) recent scoping review, including biases from training data, hallucinations referred to earlier in this section, privacy risks from leakage of sensitive information, interaction risks between workers and AI that could lead to unethical or even illegal activities, and security risks linked to threats to AI systems. Appropriate training of workers and careful monitoring of ChatGPT–human interactions in the workplace are required to mitigate these risks.

This study has several limitations. Firstly, although the theoretical framework allows for a comprehensive conceptual approach, it can also be seen as a limitation, which can be addressed in future research by conducting an external expert validation. Second, the data about WPL characteristics has been extracted from the papers that met the inclusion criteria in the scoping review. We trust that the peer review process that led to the publication of the selected papers provides a level of quality assurance. Further reassurance would imply a methodological quality assessment, which is out of the scope of this work and can be conducted in future research. Finally, our scoping review was limited by the screening of papers that had the search terms in the title, this may have overlooked relevant papers that could have provided more nuanced WPL characteristics. However, we were guided by the assumption that the papers that included the search terms in the title had a strong focus on those terms.

The paper presents three main contributions, first the identification of the contradictions between traditional and workplace learning, second the elicitation of functional requirements for WPLT, and third the analysis of ChatGPT as a technology for learning in the workplace, highlighting its benefits and limitations. We conclude that although there is a need for further development and integration of GenAI conversational agents, even in their current form, they meet a range of functionalities desirable to facilitate WPL. The findings in this study encourage a shift from work training programs based on traditional learning towards greater organisational support for technology that preserve the distinctive characteristics of WPL. Further work will include the comparative assessment of other technologies, taking ChatGPT as benchmark. This study also helps to reflect on the gap between learning management systems used in HE and the characteristics of enabling technologies for WPL. Based on the findings, we conclude that the integration of GenAI conversational agents in HE provides students with a learning experience that better resembles worker’s approach to learning; consequentially facilitating a smoother transition into the workplace and enhancing graduates’ lifelong learning skills. Further research will include study of the use of GenAI conversational agents in HE for improved graduate employability.