Understanding workplace learning dynamics using experience sampling: insights from an event-based and a time-based sampling study

Workplace learning (WPL) is crucial for individual employability and organisational success, yet effectively supporting or facilitating it remains a persistent challenge (Wijga et al., 2025). WPL encompasses all individual and social learning interwoven with work activities (Tynjälä, 2008; Wijga et al., 2025) and includes both formal and informal learning (Eraut, 2004; Marsick and Volpe, 1999). Formal WPL is planned, structured and often instructor-guided with predetermined outcomes, which are easier to document and evaluate. In contrast, informal WPL is less structured, highly situational and often embedded in everyday work activities, making it more difficult to capture (Eraut, 2004; Marsick and Volpe, 1999). However, informal learning forms a substantial part of how people develop at work and is strongly connected to performance, adaptability and innovation in organisations (Tannenbaum et al., 2009). Examples of informal WPL include looking up information, experimenting and asking for feedback (Amenduni et al., 2022; Kyndt et al., 2016). Informal WPL is a highly dynamic, context-dependent process (Tannenbaum et al., 2009), influenced by both personal (e.g. personality, education) and situational factors (e.g., task characteristics, presence and role of others and learning opportunities; Amenduni et al., 2022; Cerasoli et al., 2018; Crans et al., 2021). It can be deliberative (i.e. goal-oriented and self-regulated), reactive (i.e. spontaneously or opportunistically) or implicit (unconscious and without explicit learning outcomes; Eraut, 2004). Capturing this spontaneous and often interwoven nature of informal WPL practices in day-to-day work, therefore, remains methodologically challenging.

Partly for this reason, little is known about how individuals learn during daily work, leading organisations to miss opportunities to provide timely and individually targeted support aligned with the learners’ moment-to-moment needs (Endedijk and Cuyvers, 2022). This gap partly stems from reliance on retrospective, cross-sectional research (Rausch et al., 2022), which fails to capture the dynamic occurrences and fluctuations of how and when individuals learn (Fisher et al., 2018). Furthermore, cross-sectional designs do not provide within-person data on individual and contextual fluctuations (Eraut, 2004; Tynjälä, 2008), despite accumulating evidence that between-person, group-level data insufficiently represent individual experiences and variation in learning (Delanoeije and Verbruggen, 2020; Fisher et al., 2018; Voelkle et al., 2014). For example, Fisher et al. (2018) found that at least 32% of individual-level variance remains undetected in group-level data. As a result, organisational support is often based on aggregated assumptions and may misalign with employees’ actual learning experiences and needs.

A suitable approach of studying these processes is the experience sampling method (ESM; Myin-Germeys and Kuppens, 2022; Seifried and Rausch, 2022). ESM is an intensive longitudinal method that assesses real-time experiences, behaviours and states in daily life using repeated self-report measures. Participants typically complete brief questionnaires multiple times per day over several consecutive days in their natural environment, ensuring ecologically valid measures and minimising recall bias. This enables researchers to identify between- and within-person variation, situational influences and temporal patterns in learning processes; that is, to assess when, how and under what circumstances individuals engage in WPL (Seifried and Rausch, 2022). However, despite its potential to capture informal WPLin situ, extant WPL research has made limited use of ESM to generate process-level insights into informal learning practices over time.

A key methodological choice for applying ESM concerns the sampling approach; specifically, how often and in what way participants are prompted to report on their learning experiences. This choice directly influences the quality and type of data collected (Myin-Germeys and Kuppens, 2022). To date, however, there is limited systematic insight into how different ESM sampling design choices (e.g. prompt frequency, timing and mode) affect data quality and the insights obtained when studying WPL processes. Addressing this gap is important because it enables researchers to better align ESM designs with their study goals and advances the WPL literature by clarifying which methodological choices yield the most informative and practically actionable insights.

Two dominant sampling approaches in ESM are time-based and event-based (Rausch et al., 2022). Time-based sampling prompts participants at fixed or (semi-)random intervals a specified number of times each day. It is well-suited to capture experiences, behaviours and cognitions that occur throughout the day, such as mood and activity (e.g. learning, relaxing or procrastinating) momentary (Myin-Germeys and Kuppens, 2022). It is relatively predictable and does not rely on participant initiative, although prompts may interrupt daily flow and may not coincide with learning activities (Seifried and Rausch, 2022). In contrast, event-based sampling relies on self-initiated reporting following predefined events (e.g. WPL activities or errors), allowing more immediate assessment while minimising recall bias. However, it depends on participants recognising and consistently reporting relevant events (Rausch et al., 2022).

Both approaches strongly influence the quantity and quality of ESM data, yet these effects remain understudied, particularly in event-based designs and in the use of open-ended responses (Reiter et al., 2025; Stadel et al., 2025; Wrzus and Neubauer, 2023). Low compliance (i.e. low response rates within participants) may prevent full capture of WPL dynamics, justifying participant exclusion (Viechtbauer, 2022; Wrzus and Neubauer, 2023). For data quality, nonsensical (incoherent or irrelevant) or careless responses compromise validity and hinder meaningful insights into the WPL processes (Reiter et al., 2025). Such responses may require exclusion, which in turn affects participants’ response rates (Viechtbauer, 2022). Thus, ESM studies must be carefully designed and evaluated to ensure sufficient data quantity and quality for capturing WPL dynamics.

Various methods exist to evaluate response quality in ESM. Existing indicators for careless responses are response patterns (e.g. long-string analysis and inter-item inconsistencies) and unusually short or long response durations (Reiter et al., 2025; Ward and Meade, 2023). These techniques are primarily suited to closed-ended items. For open-ended responses, word count may serve as a partial indicator, with longer responses generally indicating higher quality, though only up to a certain point (Curtis et al., 2024; Stadel et al., 2025). However, content still requires qualitative assessment to reliably distinguish meaningful from low-quality responses (Brühlmann et al., 2020).

This study aims to advance the methodological and substantive understanding of WPL dynamics by comparing time-based and event-based ESM with closed-ended and open-ended responses. The primary objectives were to provide insights into ESM designs through evaluating compliance and data quality indicators and to explore how ESM can capture WPL as dynamic processes through within-case analyses.

This research explored how ESM captures WPL dynamics through two ESM studies using time-based and event-based sampling. Data were collected using the smartphone application of the Twente Intervention and Interaction Machine (TIIM; van ’t Klooster et al., 2024). Informed consent was obtained before participation. The Ethics Committee Behavioural, Management and Social Sciences of the University of Twente approved the study (240956).

Participants of the studies were two cohorts of 3rd year Bachelor students enrolled in a 10-week full-time challenge-based learning project taking the role of consultants. Students worked in consultancy teams on real organisational learning and development challenges with external clients at the client’s organisation and in a simulated office environment at the university, mimicking real consultancy work as closely as possible. Alongside teamwork, they engaged in self-directed professional development by formulating two personal learning goals in SMART (specific, measurable, achievable, relevant and time-bound) criteria within one of five consultancy competencies (e.g., client communication). They pursued these goals by conducting WPL experiences and reflecting on the outcomes, supported by teacher feedback. All enrolled students were invited to participate and to consent to the use of anonymised data.

Participants reported their learning activities, learning outcomes and perceived goal achievement for each WPL experience in TIIM. Firstly, participants recorded the “learning activities they engaged in to progress towards your self-development goal,” using activities adapted from Kyndt et al. (2016), for example, “consulting information sources.” Learning outcomes were measured through an open-ended reflection question. Finally, they reported their self-perceived goal achievement (“To what extent do you think you have achieved the learning goal so far?”) for the corresponding personal learning goal at that point in time on a sliding scale from 0% (“Not at all”) to 100% (“Goal achieved”). Additionally, other categorical items assessed constructs beyond the scope of the current analysis. Participants were instructed on using TIIM and completed a trial questionnaire (“log”) before data collection.

Fixed time-based sampling was applied for five weeks between November 2022 and January 2023. Participants received notifications at the start (10.00) and end (17.00 reminder) of each working day to complete one daily log, available until midnight, in which up to five WPL experiences could be reported. Each log consisted of four unique items and, per experience, five repeated items; hence, logs ranged from nine to 29 items depending on the number of learning experiences reported. Of the 25 enrolled students, 22 participated; 82% (n = 18) were women and 68% (n = 15) studied psychology.

Event-based sampling was applied for six weeks between November 2023 and January 2024. Participants reported WPL experiences in separate logs available from Monday 10.00 to Saturday 23.00, supported by notifications at 10.00 and 17.00. Logs consisted of 12 items. Of the 38 enrolled students, 33 participated; 67% (n = 22) were women and 49% (n = 16) studied psychology.

TIIM kept track of the number of learning experiences logged (i.e. response rates) and the time of logging (timestamp of every item). Compliance was assessed using the response rates. For time-based sampling, compliance is usually calculated as the proportion of completed-to-expected number of observations (Viechtbauer, 2022). Because the expected number of WPL experiences could not be determined reliably (Rausch et al., 2022), compliance was operationalised relative to a predetermined minimum number of logged WPL experiences (i.e. compliance target): eight in the time-based and 10 in the event-based study. To enable comparison between the two sampling approaches, each participant’s compliance rate was calculated as: (number of reported experiences × 100)/compliance target.

Answers to the open-ended learning outcomes item were used to assess the data quality, assuming inadequate engagement would also indicate careless responding on the closed-ended items. Consequently, the learning outcomes reflections were coded inductively and reflexively by two researchers in ATLAS.ti v25 into reflection quality. Three classifications were identified: specific (a detailed description of what was learnt, e.g. “I should try to keep a more open mind towards others’ opinions, while giving my own, and make sure that everyone understands my logic behind the choices I make”), unspecific (related to the learning activity, but without many details, e.g. “I learned to be a better coach and how to apply theory to practice”) and nonsensical (missing, seemingly unrelated or incoherent, e.g. “I am able to do that, I just need some more motivation”). After both researchers agreed on the codebook (Table S1 in the Supplementary material), one coded the time-based and the other the event-based data, as coding required consideration of the broader context (i.e. the recorded WPL experience, participant and cohort). Subsequently, they reviewed 50% of each other’s coding and discussed any notable or discrepant cases. Furthermore, word counts of the learning outcomes were assessed. Finally, response duration was calculated from TIIM timestamps, and outliers were identified using the skewness-adjusted method of Hubert and Vandervieren’s (2008).

WPL activities, outcomes and self-perceived goal achievement were traced over time to explore when, how, what and for how long participants learnt within individual learning goals.

Compliance rates and data quality indicators (reflection quality, word count and response durations) were analysed descriptively and using visualisations. Differences between sampling approaches were assessed using non-parametric tests. To better understand (changes in) data quality over time, reflection quality was explored by time of logging during the day, the week and over the period of the study, accompanied by exploratory χ²-tests. Four illustrative cases were visualised to explore possible variation in WPL trajectories. All analyses and visualisations were performed using R v4.5.1. Outliers in response durations were identified using adjboxStats of the robustbase package (Maechler et al., 2025).

In the time-based study, 22 participants reported 238 WPL experiences (mean = 10.8, SD = 4.1) across 159 logs (see Supplementary material; Figure S1). Median compliance was 125% (IQR = 100%–163%), with 4 participants (18%) reporting < 8 experiences. In the event-based study, 33 participants reported 326 experiences (mean = 9.6, SD = 4.0), and median compliance was 100% (IQR = 100%–118%), with 7 participants (21%) reporting < 10 experiences. Time-based sampling resulted in slightly higher response rates (t(44) = −1.10, p  = 0.278) and significantly better compliance (W = 251, p  = 0.037). Figure S2 in the Supplementary material visualises these frequencies.

Table 1 shows the frequencies of reflection quality in the time-based and event-based samples. Both samples had similar proportions of unspecific reflections (48% for time-based vs 46% for event-based). However, the time-based sample significantly differed with a smaller proportion of specific reflections (37%) and a larger proportion of nonsensical reflections (15%) compared to the event-based sample, χ²(2) = 6.13, p  = 0.047. Most participants showed a mix of reflection classifications (Figure S3).

Word counts correlated strongly with response duration, r_s(562) = 0.63, p  < 0.001, indicating that shorter responses also tended to be completed more quickly (Figure S5). Compared to the event-based sample, time-based responses were shorter in both word count (median = 27 vs 31 words; W = 35, p  = 0.05) and duration (median = 1.9 min vs 2.8 min; W = 25, p  < 0.001; Figure 1 and Table 1) and included fewer outliers in duration (n = 10 vs n = 24). Considerable variation existed both between and within participants (Figure S6; see the Supplementary material for additional information).

Figure 2 visualises how reflection quality relates to the other indicators. Nonsensical reflections were primarily brief (90% < 25 words, n = 56; Figure S4) and short in duration (83% < 2 min, n = 51; Figure S5). However, brief responses were not necessarily low-quality, as most were classified as unspecific (48%, n = 128) or specific 15% (n = 35). Duration-based outliers captured only 23% (n = 14) of nonsensical reflections (see the Supplementary material for additional information).

Figure 3 presents the distributions of WPL experiences by weeks, by days and within the day (see Table S2 in the Supplementary material for full frequencies). Both samples showed similar patterns by week, including lower reporting rates in week 1 and 4; the latter coinciding with the period before the Christmas holidays and a high courseload. Higher proportions of nonsensical reflections were observed in the final weeks (5 and 6) of both studies, χ²(1) = 4.36, p  = 0.037. This pattern suggests lower-quality responses towards the end of the project.

Although reporting was intended to be restricted to workdays (for time-based and Monday to Saturday for event-based), technical issues allowed entries during weekends. Half (51%) of all experiences were logged on Thursdays and Fridays, indicating a strong tendency to report towards the end of the week (55% for time-based and 78% for event-based; see Table S2). Nonsensical responses were somewhat more common in those days, although this difference was not statistically significant, χ²(1) = 2.07, p  = 0.150.

In both samples, most reports were self-initiated by participants during the workday (45% for time-based and 59% for event-based). Reminders had a small effect on frequency and reflection quality (Table S2 and Figure S7). Proportions of nonsensical reflections increased shortly after the 17.00 notification (18% in time-based vs 16% in event-based responses). In the time-based study, nonsensical reflections were particularly high in the evening (30%), whereas this pattern was less clear in the event-based sample (7%). Self-initiated responses more often resulted in specific reflections (65% for time-based and 48% for event-based), whereas nonsensical reflections occurred relatively more shortly after the afternoon reminder, χ²(1) = 14.32, p  < 0.001.

Overall, briefer and faster responses were more likely to be nonsensical and thus of lower quality. Although compliance remained relatively stable, lower-quality responses occurred more often near deadlines or outside typical working hours (Figure S8). Additional analyses on the use of these indicators for data cleaning are provided in the Supplementary material (Table S3).

All participants tracked their activities and progress of one learning goal, and 51 (93%) tracked a second goal as well. On average, the students reported more learning experiences for the first goal (median = 6, IQR = 4–7 reports) than for the second goal (median = 4, IQR = 3–6 reports). Furthermore, goals were achieved more quickly for the first goal (median = 15, IQR = 11–28 days) than for the second (median = 22, IQR = 7–30 days). No substantial differences were found between the two sampling approaches (Table S4).

Four participants with sufficiently high data quality were selected to illustrate possible variation in WPL dynamics across time-based and event-based approaches. Figure 4 presents their perceived goal achievement over time and associated learning activities for one learning goal, complemented by excerpts describing learning outcomes for contextualisation.

Anna initially showed a sharp decline in goal achievement after encountering difficulties integrating feedback. As she stated: “I started immediately and changed some texts and, in the end […] I got lost. Reflecting on it, […] while I was struggling, the revelation hit me that this is exactly why I work on my learning goal.” With this insight, she worked to overcome these difficulties and eventually reached 95% goal achievement. Her trajectory combined individual and interindividual WPL activities over three weeks.

Beate demonstrated a largely steady progression, reporting nine experiences across five days over three weeks. Her learning involved a mix of individual and interindividual activities. A notable increase from 45 to 60% followed her decision to step outside her comfort zone: “it is really important to look outside the scope of [initially provided materials] if we want to find something that relates more to specific problems.”

Casper progressed almost linearly to 100% completion within one week based on four WPL activities, with mostly unspecific reflections. Half of his activities involved information seeking. He reported a marked increase from 50% to 80% after consulting a YouTube video, which provided “approaches on how to do the preparation before making the choice.”

Dennis showed steady progression from 50% to 90% within two weeks through individual and interindividual WPL activities. After receiving peer feedback (“I learned that it seems I already made some progress as the colleague who was giving me feedback noticed it as well.”), his goal achievement increased from 65% to 90% within one week. This progress appeared to be reinforced by growing confidence and continued feedback.

These cases illustrate within- and between-person variation in WPL trajectories, activity types and sequences and progression of perceived goal achievement. Progress was not always linear, and participants differed in how quickly and through which activities they progressed towards their goals. Anna’s case highlights that learners may be confronted with challenges in their capacities but can still achieve substantial growth. The cases further suggest that the timing and sequence of WPL activities may shape perceived goal attainment, such as receiving feedback after individual engagement in Dennis’ case. Both sampling approaches proved suitable for capturing these process-oriented dynamics.

This study aimed to examine how time-based and event-based ESMs can capture dynamics of WPL by comparing indicators of data quantity and quality. The results showed that both approaches achieved good compliance, although time-based sampling produced relatively more low-quality responses. Across both approaches, low-quality responses were more likely when responses were brief (i.e. short duration and low word counts), submitted outside typical work hours or completed close to deadlines (e.g. end of week and project). The findings further illustrate how ESM can capture WPL dynamics by mapping informal learning processes over time while reducing recall bias. Together, these findings support the suitability of ESM for WPL research.

WPL is often conceptualised as either formal or informal. However, WPL experiences are better understood as configurations that vary in learning type, embeddedness in work and intentionality (Manuti et al., 2015). This study, therefore, focused on self-regulated WPL (i.e. goal-directed learning based on explicit learning goals) and captured the learning experiences participants used to achieve learning goals in a workplace setting. The student consultancy and work-integrated learning context provided a useful setting for comparing ESM sampling approaches and examining WPL dynamics. This context resembled authentic organisational work through its ambiguity, time pressure, collaboration and client interaction, requiring participants to engage and reflect on learning episodes suited for ESM. Although the findings should be interpreted in the light of this setting, the core findings (i.e. how sampling designs interact with work characteristics, such as end-of-day reporting and deadline proximity) may also be relevant in organisational settings where learning is embedded in daily work tasks and competing demands.

The findings align with previous research on data quality indicators in primarily time-based ESM studies using closed-ended items. Rapid response times were associated with a higher likelihood of careless and nonsensical answering (Reiter et al., 2025; Wrzus and Neubauer, 2023). In contrast, response-time outliers were less useful for identifying low-quality data. Consistent with previous findings (Curtis et al., 2024; Stadel et al., 2025), lower word counts appeared to be a decent indicator of low-quality open-ended items, whereas higher word counts did not necessarily indicate higher-quality reflections. Although previous work recommends removing (outliers in) rapidly completed responses (Viechtbauer, 2022), this in-depth analysis suggests that doing so would also remove many meaningful responses. Instead, these indicators could be the starting point of a more detailed data quality assessment, particularly in smaller samples and studies involving open-ended items. For example, researchers could apply an iterative cleaning procedure, in which they first exclude non-compliant respondents, then evaluate multiple indicators specific to patterns in the data (e.g. temporal patterns) and finally reassess whether participants retain sufficient valid observations.

The findings further suggest that data quality may be shaped by temporal patterns within the ESM context itself. Responses submitted outside typical working hours or close to weekly or project deadlines more often showed poorer quality. These responses may indicate increased time pressure or low-effort responses as deadlines approached, which may also affect informal WPL activities in various organisational settings. Although the deadlines were partially artificial, as participants were instructed to report at least two learning experiences per week and a minimum number overall, they also coincided with actual project deadlines. Recognising these patterns may not only inform data quality checks but also study designs by encouraging researchers to consider whether ESM participation contributes to workload burden.

Assuming participants reported WPL experiences close to when they occurred, these findings may indicate increased pressure as deadlines approached, similar to patterns observed in healthcare professionals’ professional development (Dennerlein et al., 2020). Task completion close to deadlines tends to result in poorer process and outcome quality (Balasubramanian et al., 2018). One possible explanation is procrastination, which is common in self-directed learning contexts and associated with reduced self-regulation, disorganisation and reduced cognitive and metacognitive functioning (Brandt, 2020; Howell and Watson, 2007; Rozental and Carlbring, 2014; Svartdal et al., 2020), and may have occurred within the present sample. Although this study incorporated measures to support self-regulation, such as intermediate deadlines and structured meaningful learning goals (Svartdal et al., 2020), participants still appeared to procrastinate, potentially resulting in low-effort compliance (Hailikari et al., 2021). Compared to the event-based sample, the time-based sample contained relatively more nonsensical reflections and more reports submitted outside working hours. Because event-based sampling allowed participants greater flexibility in when experiences were reported, it may have supported autonomy, increasing motivation and reducing procrastination behaviour (Oram and Rogers, 2022). More broadly, these findings suggest that ESM designs should account for workload, timing and participant burden when studying WPL processes.

The current study showed that time-based and event-based sampling had strengths and limitations regarding data quantity and quality. WPL researchers should align the design with their research questions. Event-based sampling is particularly suitable when capturing discrete learning episodes with minimal interruption, such as WPL activities, whereas time-based sampling may be preferable for assessing broader fluctuations throughout the workday, such as affect, stress or self-regulation (Seifried and Rausch, 2022). In practice, both approaches can be combined when research aims require both regular and event-specific assessment (Myin-Germeys and Kuppens, 2022).

The findings illustrate how ESM can capture bottom-up and emergent WPL processes as they unfold in daily work routines rather than through retrospective reconstruction. In line with Zirkel et al. (2015, p. 2), repeated in situ reporting enabled examination of WPL episodes across days and weeks in detail. The reconstructed trajectories suggested that learning progress was not always linear. Individuals differed in how quickly they progressed, the WPL activities they engaged in and the sequencing of those activities. For example, the results tentatively suggest that periods of more intensive learning close to deadlines were associated with lower goal achievement, indicating that high activity does not necessarily correspond to effective learning. Taking the temporal distribution of learning into account, therefore, appears important. Some learners concentrated most activities into short, intensive periods, whereas others distributed their learning more gradually over time. Importantly, the forms of learning captured were also shaped by the project context. Participants were encouraged to pursue explicit learning goals, promoting deliberative learning, while spontaneous learning opportunities may have elicited more reactive forms of learning (Eraut, 2004). Implicit learning, however, was likely less visible because it occurs unintentionally and without explicit outcomes. It likely occurred, but as a by-product of other learning forms (Eraut, 2004; Seger, 1994). These observations support the value of ESM for studying temporal and context-dependent aspects of WPL trajectories (Rausch et al., 2022; Zirkel et al., 2015). At the same time, the visualised trajectories were intended to illustrate possible variations in WPL dynamics, rather than establishing broader patterns or typologies.

This study highlights the suitability of ESM for investigating WPL dynamics and demonstrates its potential for addressing a broad range of research questions. In addition to WPL learning experiences, ESM may be used to investigate self-regulatory processes, including goal-setting, affect, motivation and adaptation, in an ecologically valid way (Endedijk and Cuyvers, 2022). Previous literature already provides guidance on research questions for ESM in general (e.g., Kuppens and Myin-Germeys, 2022) and for intensive longitudinal research in WPL specifically (Rausch et al., 2022). The current study demonstrates that open-ended items concerning WPL outcomes are also feasible. This creates opportunities for new research questions, for example, regarding reflection quality. While this study explored reflection specificity as an indicator of ESM data quality, future research could develop more rigorous analytical approaches to assess reflection depth and thereby enrich understanding of WPL dynamics. Beyond ESM, other ecological momentary assessment modes may also offer valuable insights into WPL dynamics. For example, researchers could incorporate passive sensing methods, such as digital phenotyping (smartphone-based behaviours, GPS tracking) and wearable sensors (e.g. accelerometery, skin conductance, electrocardiography). These approaches provide continuous, unobtrusive and objective measures of behaviours, environments and physiological processes (Hachenberger et al., 2023; Raugh et al., 2019; Ren et al., 2023; Weber et al., 2022). Investigating WPL dynamics in daily life may create opportunities for more tailored and timely support and interventions. ESM and related intensive longitudinal methods may help determine when intervention is most needed, for example, to improve emotion regulation in the workplace (Zhu et al., 2025) or enhance workplace training programmes (Johansson and Andersson, 2022). Finally, future studies should replicate these findings in organisational settings to further examine compliance and data-quality patterns across different work and learning settings and to establish context-appropriate guidance for ESM designs in WPL research.

This study demonstrates how WPL can be investigated using both categorical and open-ended items, and how these relate to data quantity and quality. It also contributes to the predominantly quantitative ESM literature by incorporating a more qualitative perspective and by examining sampling approaches, compliance and data-quality indicators. Exploring open-ended items enabled a more comprehensive assessment of data quality. One underlying assumption was that adequate engagement with the open-ended item would also indicate adequate responses to other (categorical and continuous) items. Although this assumption could not be systematically tested using longstring and inconsistency analyses (Ward and Meade, 2023) due to a lack of suitable items, the findings generally support it. Some WPL experiences appeared to contain incorrectly specified activity types that did not fully align with the reported learning outcomes. However, these inconsistencies were infrequent and typically reflected less fitting rather than random or nonsensical activity types. Furthermore, no unusual patterns emerged in perceived goal achievement scores, suggesting that participants were generally not careless or intentionally incoherent in responding.

Several other limitations need to be addressed. Coding data quality at this level of detail is difficult to replicate and time-consuming, especially in large data sets. The coding was exploratory and explicitly aimed to derive a parsimonious data quality indicator from open-ended responses to support comparison between sampling approaches, rather than to establish a validated construct. Although the lack of a formal interrater reliability assessment limits claims about reliability, the consistent, cross-reviewed application of broad, mutually exclusive categories allowed for cautious comparative interpretation.

Additionally, the use of a student consultancy sample limits the generalisability to organisational settings. Although the project incorporated several workplace characteristics, students differ from employees in important respects, such as formal roles, hierarchical structures, performance pressures and consequences and embeddedness in ongoing work relationships (Landers and Behrend, 2015). These contextual differences may influence both self-regulated WPL and participation in ESM studies. Employees may have more autonomy to engage in WPL, potentially resulting in learning activities that are more aligned with their interests. Moreover, the presence of imposed deadlines and minimal engagement requirements may also have reduced the reflection quality and increased careless or socially desirable responding (de la Croix and Veen, 2018), more than would be expected in a context where employees are intrinsically motivated to learn. At the same time, previous studies have shown that employees also do not always engage in self-regulated professional learning (Cuyvers et al., 2024). Although the use of a student sample is not problematic in itself, this choice should be considered in relation to the research question being addressed. Consequently, the observed compliance and data-quality patterns may not directly transfer to organisational contexts. Future research should therefore replicate these findings in workplace settings.

Furthermore, although this study refers to time-based and event-based sampling, neither fully reflects their typical forms. The time-based design more closely resembled a daily diary approach (Rausch et al., 2022), whereas the event-based design uncommonly included reminder notifications because these could not be disabled in TIIM at the time of data collection. Within the context of this project, these reminders were considered beneficial because they supported students’ self-directed learning.

Finally, the unequal participant and observation counts between studies complicate direct comparison between sampling approaches. These differences may partly reflect the sampling designs, but may also stem from unmeasured factors (e.g. willingness to reflect) or random variation. Notably, no significant differences emerged between the two groups in the distributions of learning activities, suggesting that systematic content-level differences were limited. Furthermore, because ESM research is usually explorative and does not necessarily aim to produce fully representative samples, these differences may be less problematic. Nevertheless, the findings should be interpreted cautiously.

This paper demonstrates how ESM can be used to generate process-oriented insights into WPL dynamics while examining how sampling designs influence data quantity and quality. Although conducted in a student WPL context, these findings support the broader potential of employing intensive longitudinal approaches for studying and potentially facilitating learning as it unfolds in daily work.

Gerko Schaap and Nick Goossen contributed equally to this work.

The supplementary material for this article can be found online.