Discovering students' continuous intentions to use ChatGPT in higher education: a tale of two theories Free

ChatGPT, a generative AI (GenAI), has significantly impacted students’ academic lives by providing intellectual support and access to a vast knowledge base, aiding in content production (Bin-Nashwan et al., 2023). ChatGPT accelerates students’ learning processes (Liu and Ma, 2023), which is especially helpful in circumstances where accuracy and speed are crucial (Ullah et al., 2024) by posting queries and obtaining instant access to information (Zeb et al., 2024) and encouraging intellectual development and achievement (Yu, 2023) in higher education institutions (HEIs). Moreover, it reduces stress in academic environments (Bin-Nashwan et al., 2023), especially for those struggling with time management and task completion. Thus, understanding ChatGPT’s aspects, including students’ satisfaction and intent to use GenAI-powered scholarly assistants, is crucial for improving educational experiences.

Several studies have surfaced by acknowledging the significance of ChatGPT in education (i.e. Bin-Nashwan et al., 2023; Kasneci et al., 2023) and examining the factors that influence university students’ intentions to use ChatGPT (i.e. Strzelecki, 2023). From these studies, Foroughi et al. (2023) found that intention to use ChatGPT is significantly influenced by performance expectancy, effort expectancy, hedonic motivation, and learning value by employing the UTAUT2 model (Unified Theory of Acceptance and Use of Technology). Similarly, Strzelecki (2023) also uncovered that social influence, habit, and personal innovativeness are positively associated with students’ intentions.

Previous research has utilised UTAUT, UTAUT2, and TAM (Technology Acceptance Model) to examine students’ continuous intentions to use ChatGPT (e.g. Duong et al., 2023; Strzelecki, 2023). However, most of these studies have looked at theories separately, investigating them distinctly within the education perspective. Likewise, this research seeks to close the gap by offering empirical evidence of university students’ intentions to use ChatGPT continuously, predominantly through incorporating two frameworks, i.e. the UTAUT3 and ISS (Information Systems Success) models. Doing so can help understand the factors influencing students’ satisfaction with continuous intentions to use ChatGPT. Also, to shed more light on students’ satisfaction, this study expands the existing ISS model to include conversational quality (Eysenbach, 2023). Hence, this research addresses the research questions:

This study aims to understand students’ acceptance and continuous use of ChatGPT and predict future trends in AI-powered tools in HEIs. This study would also assist in epitomising the dynamic interaction between university students’ satisfaction with using ChatGPT to maximise its potential as a powerful learning tool, as seen by the extended ISS model, and their ongoing plans to utilise it, as elucidated by UTAUT3. The results guide developers in enhancing educational technology, including GenAI systems, and can help educationalists integrate ChatGPT into HEIs’ practices, enhancing its acceptance and application.

One well-known theoretical framework that has attracted interest in technology acceptance and adoption is UTAUT3. To further extend the research on UTAUT2 (Venkatesh et al., 2012) and to give a comprehensive grasp of the factors impacting users’ adoption and usage of new technology, Farooq et al. (2017) developed UTAUT3, an expansion of the original UTAUT model (Venkatesh et al., 2003).

UTAUT3 incorporates factors from well-known models, such as TPB (Theory of Planned Behaviour), TAM, and UTAUT. Thus, by joining these models, UTAUT3 provides a comprehensive viewpoint of the influencers of new technology adoption. Because of its flexibility and adaptability, it is especially well-suited to study students’ acceptance of technology in educational contexts, such as AI-driven applications (Foroughi et al., 2023; Strzelecki, 2023), which has been extensively studied using this model. Thus, by integrating UTAUT3, this study can examine distinctive features of the GenAI tool, i.e. ChatGPT, in HEIs by investigating factors impacting students’ intent to use it.

Performance expectation has repeatedly been a strong predictor of users’ intentions to continue using technology, according to the UTAUT (Venkatesh et al., 2012). Performance expectancy encourages students to use tools efficiently and accomplish tasks quickly. Likewise, university students are more likely to establish the intention to use ChatGPT when they see it as a beneficial tool for improving academic performance (Foroughi et al., 2023). Thus, this link emphasises the significance of students recognising ChatGPT as a valuable GenAI tool to encourage consistent use. The following hypothesis is suggested:

Effort expectancy emphasises that users’ intent to use GenAI technology regularly is critically induced by impressions of its usability (Bernabei et al., 2023). Because reduced perceived effort translates into increased user acceptability to continue using the GenAI tool, students who believe ChatGPT to be user-friendly are more likely to interact with it regularly. Consequently, increased effort expectation makes it easier for students to use ChatGPT, supporting continued usage and consistent with UTAUT’s projections (Duong et al., 2023). The following hypothesis is proposed:

Al-Adwan and Al-Debei (2023) reveal how social groupings, such as peers, lecturers, and family members, significantly impact students’ adoption and continuous intentions to use technology, which contends that users want to emulate the attitudes and practises of their reference groups. Students naturally want to fit in with their reference groups and be accepted; they are likelier to use ChatGPT if they believe it is a valuable tool that their peers support. Therefore, social influence affects students’ intent to utilise ChatGPT because it meets their psychological demand for social approval. The hypothesis is formulated below:

Users’ intent to keep using technology is influenced by hedonic motivation, which reflects intrinsic pleasure from utilising technology (Alkhwaldi, 2023). Foroughi et al. (2023) emphasise the role of hedonic motivation, such as conversational pleasure, competence, and interaction, fostering students’ contentment. Students who find such GenAI tools, i.e. ChatGPT, to be entertaining and emotionally rewarding are more likely to acquire a desire to continue using it, as technology utilisation is motivated by both utilitarian and hedonic demands. Consequently, incorporating hedonic incentives into this study is consistent with users’ innate desire for delightful experiences, influencing their intention to continue using ChatGPT. This formulates the following hypothesis:

Learning values are critical in determining university students’ intent to keep using technology, which measures the advantages of technology to educational development (Foroughi et al., 2023). Chiu (2021) underlines that students prefer to participate in activities that satisfy their core psychological requirements, such as the need for competence, autonomy, and relatedness. ChatGPT meets intrinsic needs and supports learning goals; students are likelier to stick with it, which helps them succeed academically (Foroughi et al., 2023). It is essential to incorporate learning value, as it captures the psychological forces that underlie students’ ongoing intent to use ChatGPT. This directs the following hypothesis:

Habit represents the gradual automation of users’ behaviours (Tan, 2023; Tan et al., 2022); habits are formed when action is repeated consistently, such as using ChatGPT. Habits lower cognitive effort and raise the probability of repeating behaviour (Venkatesh et al., 2012). University students who regularly use ChatGPT are prone to stick with it since their intentions are driven more by the automatic nature of the behaviour (Venkatesh et al., 2023). Subsequently, habit captures the psychological foundations of students’ ongoing intent to utilise ChatGPT as it becomes a habitual aspect of their academic lifestyle. This proposes the following hypothesis:

Walle et al. (2023) highlight how the availability of resources and technical assistance increase the practicality and ease of using technology. Students who believe that ChatGPT is backed by sufficient resources and technical support and compatible with other tools are prone to stick with it since these favourable conditions lower obstacles and make utilising the platform more accessible (Foroughi et al., 2023). Hence, by facilitating ChatGPT’s adoption process, facilitating conditions boost students’ ongoing intent to use ChatGPT. It is proposed that:

Al-Adwan et al. (2023) emphasise the tendency for people with higher levels of personal innovativeness to be eager to try new technology. Hence, students with greater personal innovativeness will likely stick with ChatGPT because they are naturally curious about it and readily accept it (Foroughi et al., 2023). This study incorporates personal innovativeness, capturing students’ psychological propensity to adapt, affecting their ongoing intent to utilise ChatGPT. The following hypothesis is suggested:

Developed by DeLone and McLean (1992), the ISS model stands out as one of the most significant theories for elucidating the triumph of information systems (IS). It comprises three quality factors: system quality, information quality, and service quality (DeLone and McLean, 2004). These factors are used to gauge IS’s technical and semantic achievements. The model suggests that the quality of the system, the information it provides, and the services it offers impact user satisfaction and willingness to use it. The model has become predominant for assessing quality aspects of technologies, including e-learning (Hii et al., 2023).

System quality is the assessment of favourable attributes of an IS that employ perceived ease of use, system functionality, response speed, and adaptability for evaluation (Al-Fraihat et al., 2020). This study pertains system quality to the degree to which ChatGPT is deemed “user-friendly” and can be utilised without difficulties. For instance, Al-Fraihat et al. (2020) found that the quality of the educational system positively influences user satisfaction and indirectly influences the utilisation of the system. Educational elements in e-learning systems, such as collaborative learning tools, can lead to increased user satisfaction and usage optimisation. Alzahrani et al. (2019) also reported that system quality predicts students’ satisfaction with digital library systems. Therefore, ChatGPT’s system quality is assumed to be vital in affecting students’ satisfaction. The following hypothesis is proposed:

Information quality is typically assessed by evaluating the content-related output in terms of timeliness, accuracy, reliability, and trustworthiness (DeLone and McLean, 2004). In this study, information quality is defined as the correctness, usefulness, and timeliness of the content produced by ChatGPT. Assessing information quality is crucial in evaluating the effectiveness of information in achieving educational objectives and the substantial issues that arise from subpar information quality (Al-Fraihat et al., 2020). Previous research has established a significant relationship between system information quality and user satisfaction (Alzahrani et al., 2019). Hence, this research suggests that if ChatGPT consistently delivers high-quality information to students, it will likely enhance their satisfaction with the system. A hypothesis is formed below:

Service quality is evaluated by examining the level of assistance provided by IS. In this research, service quality can be effectively gauged by assessing ChatGPT’s competence, follow-up support, empathy, confidence, reliability, and responsiveness. Previous research has shown that service quality significantly enhances satisfaction with e-learning (Al-Fraihat et al., 2020). As a result, it can be inferred that if ChatGPT is easily accessible, user-friendly, dependable, efficient, and adaptable, students are more likely to feel satisfied with it. This suggests the following hypothesis:

Conversational quality in AI-mediated communication influences users’ satisfaction significantly (Hsu and Lin, 2023). High conversational quality in ChatGPT, typified by responsive and courteous interaction, lessens user annoyance and raises perceived efficacy (Lee et al., 2021). Students will be satisfied when they view ChatGPT as being able to comprehend and answer in a way that mimics human communication. Therefore, users are more likely to be satisfied with ChatGPT when interactions are marked by prompt and contextually relevant dialogues, which raise students’ engagement. The following hypothesis is suggested:

Positive attitude is correlated with user satisfaction when using technology, whereby users’ desire to continue is driven by instant gratification (DeLone and McLean, 2004). Consequently, when university students are contented with the support provided by ChatGPT, their favourable perceptions are more likely to result in a desire for usage that goes along with contentment, needs, and expectations of an optimistic user experience (Rajeh et al., 2021). Based on correlations between attitude and behaviour in psychology, user satisfaction is crucial in influencing continuing intent (Alrawabdeh et al., 2023; Rasheed and Nandukrishna, 2023). The hypothesis is developed below:

The research model is exhibited in Figure 1.

Data were gathered through an online survey (Google Forms). The research’s target demographic is university students, as the objective is to investigate factors influencing students’ continuous intentions to employ ChatGPT. Before initiating data collection, ethics clearance was obtained from one of the authors’ universities. Informed consent was ensured, provided that respondents voluntarily participated in the research. The questionnaire was pre-tested by three academic experts to guarantee the face validity of every scale being studied. The authors disseminated the survey among students at one of the top private universities in Malaysia. In two weeks, 388 useable responses were accumulated, with a gender distribution of 198 females, 174 males, and 16 individuals who chose not to disclose gender. Two hundred forty-nine (249) respondents were between 18 and 20 years old, while the remaining respondents were older than 20, indicating a significant presence of young adults.

All scales were adapted from previous studies (See Table 1). All items were measured on a 5-point Likert scale ranging from strongly disagree (1) to strongly agree (5).

Partial least squares structural equation modelling (PLS-SEM) is employed because it is well-suited for an exploratory nature, aligning seamlessly with the primary goal of this research (Zeb et al., 2023), which is to integrate the UTAUT3 and ISS models to predict students’ intentions. Second, PLS-SEM empowers the research model’s predictive capacity. Lastly, its flexibility in accommodating various data assumptions is an additional.

Common method bias was examined using a full-collinearity test. This test was executed by performing regression analysis on all independent and dependent variables using a dummy variable generated from random numbers (Kock, 2015). The result showed that all variance inflation factors were less than 5. Next, this study examined convergent and discriminant validities. The internal consistency of measures was achieved based on Cronbach’s alpha and composite reliability, with both greater than 0.7. Besides, Table 2 shows that all items have a factor loading exceeding 0.7 and AVE greater than 0.5, indicating good evidence of convergent validity. This study employed the Heterotrait-Monotrait ratio inference method (HTMT) to ensure discriminant validity. Discriminant validity is confirmed if the HTMT value is less than 0.9 or the bootstrapped upper limit is significantly below one (Henseler et al., 2015). As seen in Table 3, although some HTMT values were greater than 0.9, the bootstrapped upper limit values were less than 1. Hence, it can be safely concluded that this study confirmed sufficient reliability and validity.

Path coefficients are evaluated via the bootstrapping procedure, in which the significance is examined (see Table 4) using t-values, p-values, and confidence intervals. Hedonic motivation (β = 0.163, p < 0.001), habit (β = 0.151, p < 0.001), facilitating conditions (β = 0.112, p < 0.05), and satisfaction (β = 0.529, p < 0.001) have significant positive influence on continuous intentions. However, performance expectancy (β = 0.063, p > 0.05), effort expectancy (β = −0.065, p > 0.05), social influence (β = −0.032, p > 0.05), learning value (β = 0.056, p > 0.05), and personal innovativeness (β = 0.016, p > 0.05) do not have a significant influence on continuous intentions. Next, it was found that system quality (β = 0.156, p < 0.01), information quality (β = 0.269, p < 0.001), service quality (β = 0.245, p < 0.01), and conversational quality (β = 0.272, p < 0.01) have significant positive influence on satisfaction. The coefficient of determination (R²) values for continuous intentions and satisfaction were 0.739 and 0.767, respectively.

We also performed PLSpredict analysis (see Table 5), a procedure for the out-of-sample prediction that “involves estimating the model on an analysis (i.e. training) sample and evaluating its predictive performance on data other than the analysis sample, referred to as a hold-out sample” (Hair et al., 2019, p. 12). It allows researchers to examine the predictive capability of the PLS model. PLSpredict performs k-fold cross-validation, where each subset of the entire dataset is referred to a “fold”, and k indicates the number of such subsets. The dataset is divided randomly into k equally sized data subsets, with k set to a value between 5 and 10 and r to 10. In other words, in k-fold cross-validation, the algorithm divides the entire dataset into k equally sized subsets. Each subset (hold-out sample) is then predicted using the remaining k-1 subsets, which together form the training sample. The results revealed that all indicators’ RMSE and MAE values generated were lower than the naıve LM benchmark. Hence, the model demonstrates a strong predictive capability.

This study found that performance expectancy does not significantly influence university students’ continuous intentions to utilise ChatGPT, which disagrees with earlier research (Al-Emran et al., 2020; Chiu and Wang, 2008; Huang and Chueh, 2022). A plausible rationale is that students have optimistic expectations of ChatGPT performance while using it in their studies, such as report writing, but their grades may not meet their expectations. Another possible reason is that since current students are technology-savvy, performance expectancy is no longer their concern. As a result, determining the substantial effect of continuous use is difficult. Moreover, the students may use it for entertainment and fun instead of academic pursuits, which may impact the role of performance expectancy (Shaqrah and Almars, 2022).

The results revealed that effort expectancy does not significantly influence continuous intentions, consistent with earlier research by Mohd Rahim et al. (2022), which mentioned that students indicate that it is easy to use the tools to answer their inquiries; hence, effort expectancy is not their concern. The current university students have solid technological literacy and may find ChatGPT too easy to use, reducing the influence of effort expectancy on their continuous use intentions.

This study also uncovered the insignificant result of social influence and continuous intentions. One probable rationalisation may be due to the background profile of respondents, who are university students and have ample experience with technology. Hence, the impact of social influence is reduced (Boontarig et al., 2012). Another possible reason is that using ChatGPT is not officially encouraged by HEIs; some lecturers even ban students from using it for their assignments. Therefore, the use of ChatGPT may be considered a private action by the students; this may limit discussion about it among the students and on social networks.

This study proved that hedonic motivation is positively linked to university students’ continuous intentions to use ChatGPT. University students think ChatGPT is entertaining and emotionally rewarding. They are attracted to the conversation with ChatGPT, and they feel having a conversation with ChatGPT is fun, enjoyable, and exciting. Moreover, the students enjoy searching for information more if ChatGPT recommends it. The intrinsic pleasure obtained from ChatGPT fosters the students’ contentment and sustained engagement. This result is consistent with existing research indicating that the use of technology, such as the GenAI tool, is influenced by hedonic motivation (Alkhwaldi, 2023; Foroughi et al., 2023), which leads to a strong desire to continue using the technology.

Also, this research found that learning value does not affect students’ continuous intentions to use ChatGPT, which indicates that using ChatGPT cannot help students succeed in their studies. A possible explanation may be that ChatGPT has limited academic relevance. For example, ChatGPT cannot scrutinise research articles from bibliographic databases, textbooks, conference papers, and other educational resources in the academic field. Subsequently, other reasons may be due to ChatGPT not providing updated literature and appropriate references (Al-Worafi et al., 2023), conversational AI models cannot guarantee factual accuracy (Zhao et al., 2023), and the answers from ChatGPT may be accompanied by misleading information (Tlili et al., 2023). Hence, university students view ChatGPT as less effective than traditional academic sources for their assessment writing, which reduces the impact of learning value on ChatGPT.

Habit was found to significantly influence continuous intentions, consistent with prior studies (Strzelecki, 2023; Venkatesh et al., 2023). This result demonstrated that the use of ChatGPT in the learning process has become innate to university students who intend to use ChatGPT more regularly. The reason is that the students who repeatedly use ChatGPT for their educational pressures are prone to stick with it since their intents are driven more by the automatic nature of their behaviour than by their conscious thoughts.

This study’s results indicate that facilitating conditions influence university students’ continuous intentions to use ChatGPT, consistent with past scholarly research by Foroughi et al. (2023) that found that students think they have the necessary resources and knowledge to utilise ChatGPT. If they face difficulties while using ChatGPT, getting help from their peers is easy. Besides, since ChatGPT is compatible with other AI-supported digital tools such as WordTune and Grammarly that they use to improve their assessment writing (Perkins, 2023), coupled with the availability of resources, technical assistance, and compatibility, it makes utilising ChatGPT more accessible, which boosts students’ ongoing intent to use ChatGPT (Bin-Nashwan et al., 2023; Duong et al., 2023; Tiwari et al., 2023).

Despite the above findings, the insignificant result of personal innovativeness may be because current students are frequently exposed to a wide range of AI-based technologies. They might be using ChatGPT because of its simplicity in generating answers to their inquiries (Kasneci et al., 2023) and not because of their curiosity and psychological propensity to adapt to this emerging GenAI technology.

The findings revealed that system quality positively affects students’ ChatGPT satisfaction. Coherent with prior research (Al-Fraihat et al., 2020), system quality is about perceived ease of use, system functionality, response speed, and adaptability. This result proved that students revealed that ChatGPT is user-friendly, has a good response time, is highly reliable with minimal downtime, and can be used easily. Hence, their satisfaction with ChatGPT and its optimised usage is high.

Next, information quality was found to influence satisfaction significantly. This finding aligns with previous research (Alzahrani et al., 2019), which reported that ChatGPT must provide up-to-date and sufficient information relevant to the student’s needs. If ChatGPT consistently delivers high-quality information to students to fulfil their educational purposes, it will likely enhance their satisfaction.

Furthermore, this research found that service quality positively influences satisfaction, which aligns with previous research (Al-Fraihat et al., 2020) stating that service quality significantly enhances satisfaction with e-learning. The students echoed that ChatGPT has visually attractive interface to increase satisfaction and provide correct, personalised, and instant responses to students’ inquiries, eventually fulfilling their learning needs.

Moreover, conversational quality was discovered to affect students’ satisfaction positively. The result is consistent with earlier scholars (Hsu and Lin, 2023) who have mentioned that conversational quality in AI-mediated communication, such as responsive and courteous interaction, can increase users’ satisfaction. For instance, ChatGPT can improve students’ satisfaction by consistently and competently giving prompt responses.

This research proved that university students’ satisfaction positively affects their continuous intentions to use ChatGPT. While the students believe that ChatGPT can meet learning expectations and fulfil learning needs, they are more willing to continue using ChatGPT more often than other alternative means in the future. The result is consistent with previous research, which has emphasised the positive relationship between user satisfaction and continuing intents, whereby users’ favourable perceptions of the GenAI technology are more likely to result in a desire for usage that goes along with their contentment, needs, and expectations of an optimistic user experience (Rajeh et al., 2021).

This study revealed that ChatGPT is powerful, and university students must use it cautiously. With this, HEIs and policymakers must understand students’ perspectives while integrating ChatGPT into the educational process.

As habitual use and hedonic motivation strongly influence students to use ChatGPT regularly, HEIs and policymakers need to adapt new teaching pedagogies, for instance, by adjusting teaching and assessment practices, particularly in accommodating to the recent reform in HEIs when it comes to integrating ChatGPT efficiently with curriculum, matching it with specific learning objectives. Besides, HEIs need to explore the correct use of ChatGPT further and provide students with the necessary guidelines to use ChatGPT more ethically. Students must know how to use ChatGPT with integrity and understand their roles and responsibilities as digital citizens. With this, comprehensive workshops can be provided by HEIs to teach students about the ethical and appropriate usage of GenAI technologies. In terms of supporting facilitating conditions, HEIs can give technical support to help students become proficient users of ChatGPT and offer onboarding programmes that educate students on the tool’s possibilities. Understanding the factors influencing students’ continuous use intentions on ChatGPT can help tailor GenAI technology usage for better educational outcomes in HEIs. Likewise, the practical implications could be seen in how the factors (i.e. hedonic motivation, habit, and facilitating conditions) influence HEIs’ current and future landscape, particularly since AI technologies become regular in students’ daily lives.

Moreover, the factors that affect students’ satisfaction with ChatGPT (i.e. system quality, information quality, service quality, and conversational quality) can also guide the enhancement of ChatGPT for more advanced digital competence among students. These outcomes help developers understand the features of GenAI tools by considering their responsiveness, relevance, and student experience to improve performances by ensuring the accuracy and usefulness of information, delivering high-quality service, and improving conversational expertise.

The study revealed factors that affect university students’ continuous intentions to use ChatGPT and factors that affect students’ satisfaction with ChatGPT by integrating the UTAU3 and extended ISS models. The empirical results proved that hedonic motivation, habit, and facilitating conditions positively influence students’ continuous intentions to use ChatGPT. System, information, service, and conversational quality positively impact students’ satisfaction with ChatGPT. A solid and positive relationship exists between students’ satisfaction with ChatGPT and their continuous intentions to use it.

Some limitations should be acknowledged and further researched. Firstly, cross-sectional data were collected and did not capture changes in students’ perspectives over time. Furthermore, ChatGPT is constantly evolving, and with the advent of GPT-4, factors influencing students’ satisfaction and continuous use intentions might change. However, despite these limitations, this research provides a solid foundation for understanding factors affecting students’ use of ChatGPT in their learning process. Future research can focus on longitudinal studies to track changes in the factors influencing students’ use of ChatGPT over time. Additionally, investigating from educators’ perspectives can identify strategies for addressing potential risks, challenges, and drawbacks of GenAI technologies to achieve more responsible educational outcomes. Lastly, data were collected from students at a leading private university in Malaysia. Therefore, the findings of this study may not be generalisable to the broader population.

We would like to thank each participant who helped make this research possible. Thank you for all of your valuable time and constructive feedback.

Funding: This research received no specific grant from funding agencies in the public, commercial, or not-for-profit sectors.