In the era of artificial intelligence, natural language processing (NLP) models are revolutionizing numerous sectors. This research aims to explore the perceived value of them among university students. In particular, it aims to investigate how gender may influence students’ intention to use these models in educational contexts, highlighting potentially significant differences that could inform the implementation and adoption of educational technologies.
This study investigates the relationship between perceived value and students' intention to adopt NLP models, considering gender as a moderator. The research involves 562 students from the University of Salerno, in Italy, and uses confirmatory factor analysis to evaluate the reliability and validity of the measurement scales. A regression model with robust errors is used to explore the moderating role of gender on the relationship between perceived value and intentions of use of NLP models.
The results reveal a significant positive association between perceived value and intention to use NLP models, confirming that students with higher perceived value are more likely to adopt these technologies. Furthermore, gender moderates this relationship, indicating that females are less prone to use NLP models than male counterparts.
Research takes on a significant role in the academic field, underlining the importance of adapting teaching practices to the increasingly widespread digitalization. The inclusion of NLP models in university programs emerges as a possible improvement of the learning experience, ensuring cutting-edge education in tune with the needs of the digital society.
Introduction
The launch of Chat Generative Pre-Trained Transformer (ChatGPT) in November 2022 by OpenAI fascinated and, in a certain sense, “captured” the world, so much so that over a million users subscribed within just a week. Instagram, Facebook, Twitter and Netflix took 75, 300, 720 and 1,200 days, respectively, to reach this number of users (Firat, 2023). ChatGPT is a revolutionary model of Artificial Intelligence (AI), based on large language models, a branch of AI, developed starting from Natural Language Processing (NLP) (Filippo et al., 2024; Kanbach et al., 2023). This is a language in which statistical models based on probabilistic structures replace previous models based on systems of rules of computational linguistics. Thanks to an unsupervised or partially audited machine learning framework, the chat generates output using statistics and probabilities. It is an NPL model trained on a large set of data and texts taken from the internet, so as to predict the next word in a sentence, generating output similar to human ones as an answer to a question (Kumar et al., 2024; Roumeliotis and Tselikas, 2023).
ChatGPT is now in its fourth version and is not the only NLP tool available. This universe is large and above all constantly evolving; each model has unique characteristics that make it suitable for performing specific tasks or outlining scenarios inherent to particular fields, including education (Choi et al., 2023; Duong, 2022) Therefore, innovation manifests itself through constant refinement of both language comprehension and generation capabilities, ensuring that models are increasingly efficient and suited to the specific needs of end users (Ambele et al., 2022; Ozdamli and Cavus, 2021).
As it is easy to imagine, learning models need to transform and evolve in harmony with the progress of the Knowledge Society. Scientific research shows how technologies improve the effectiveness of teaching and adapt to different learning styles (Liu et al., 2020; McGrath et al., 2017; McGrath and Åkerfeldt, 2019). However, the world of education is rather phlegmatic in adopting new techniques due to cultural, economic and technical resistance (Liu et al., 2020).
Most university education is still based on textbooks, lectures and discussions, whereas traditional learning activity is limited to listening, memorizing and note-taking (Ngwoke et al., 2022). In fact, it focuses on the transfer of contents attributable to a single discipline and is generally entrusted only to a teacher or expert. It is based on the mere acceptance of the concepts transmitted and on the measurement of results through scores, tests and aptitude for the transfer of the information received, as well as the use of texts, readings and presentations with an impetus towards individual work and competition (Quaglino, 2005). Over time, however, the need has emerged to implement a more advanced learning model centred on the depth of understanding of content, mastery of concepts and problem-solving ability; to integrate learning with personal activities and give emphasis to the interests and predispositions of the learner, also through the support of the teacher; to carry out an evaluation based on progress and results achieved over time, using different learning sources such as interviews, documents, data or materials developed by the student, as well as encouraging teamwork and collaboration (Testa, 2023). Furthermore, it emerges from the literature that tutoring schemes must also strengthen the active participation of learners, promoting contexts suitable for carrying out personalized learning (Zhai, 2023).
The use of NLP models can help broaden the student's knowledge, freely explore innovative ideas and resolve doubts quickly (Cimino et al., 2024). The transformation of learning models, favoured by the use of technological tools such as ChatGPT, must therefore be understood as an effort in the direction of improving the quality of education (Cotton et al., 2023; Sari and Yudha, 2022). NLP models are trained on a huge amount of data ranging from scientific articles, online texts, news blogs, textbooks, specialized forums to informational Web pages. This allows us to effectively respond to various types of information requests (Graf and Bernardi, 2023; Kasneci et al., 2023). The capacity of human language makes such tools useful for writing tasks, such as essays, short stories, literary summaries and broadsheet or marketing content (Patel and Lam, 2023).
Consequently, to understand how the education sector can make use of NLP models, we believe that first of all it is necessary to explore the value that students attribute to such models and, by virtue of this perception, investigate their intention to adopt aforementioned technology and analyae whether gender differences can influence this relationship.
Some studies have examined the role of gender differences in the propensity to use technological tools, arriving at different results. However, some research shows that women feel less comfortable than men in using digital tools (He and Freeman, 2010). More recent articles demonstrate that men are more likely to adopt AI tools (Tubadji et al., 2021).
Based on these premises, it was deemed appropriate to analyse the value perceived by university students in relation to the new NLP models in the educational field to understand the propensity for their use as well as the role that gender difference has in this relationship.
Literature review
The history of technology, rooted in the Greek term τέχνη (techne), meaning “art, skill, craft,” has traversed a complex path of evolution over the centuries. Initially, the concept was limited to the art or skill of performing certain actions, but over time it has enriched itself, incorporating not only human skills but also the tools and processes used to achieve specific goals (Nightingale, 2014).
In the context of scientific literature, technology is defined as the set of methods, systems and devices derived from scientific knowledge and used for practical purposes (Houkes, 2009; Troisi et al., 2021). Dmitry Lvov and Sergey Glazyev have contributed to defining this evolution through the concept of a “technological paradigm,” identifying five fundamental stages in industrial development (Almgren and Skobelev, 2020). These paradigms include the era of mechanization, the railway era, the widespread use of electricity as a primary energy source, energy engineering and machinery production and finally the emergence of computer science and industrial biotechnology (Almgren and Skobelev, 2020) Within these paradigms, significant impacts on social and economic systems have been observed, culminating in the embrace of biotechnology, nanotechnology and AI (Almgren and Skobelev, 2020), leading us to the present day.
This succession of technological paradigms has deeply permeated every aspect of human life, including education, dividing the impact of technology into four categories of interest: technology as artefacts, as knowledge, as activity and as a facet of humanity (Jones et al., 2013; Lind et al., 2023). Technology, the product of human ingenuity, has the power to transform classrooms into dynamic learning environments: devices such as computers and educational software actively engage students, not only as tools but also as an expression of human activity (Jones et al., 2013; Visvizi et al., 2023). Recent research indicates a growing use of technology by students, as it fosters more interactive and effective learning (Raja and Nagasubramani, 2018; Troisi et al., 2023; Visvizi et al., 2023) facilitating rapid and efficient knowledge transfer as a result (Ozdamli and Cavus, 2021; Visvizi et al., 2019). Consider online research, video calls via platforms like Zoom, Microsoft Teams, Google Meet and access to webinars on topics of interest.
In particular, over the past two years, there have been significant advances in digital tools that have spurred a rapid expansion of AI technologies, including NLP (Bahroun et al., 2023; González García et al., 2017). Specifically, the use of AI and chatbots in learning has attracted the attention of researchers, and the need to investigate students' attitudes towards NLP models in education has emerged (Pavlik, 2023; Rospigliosi, 2023; Wu and Yu, 2023).
NLP models in the educational field have been designed to facilitate, enhance and personalize the learning experience (Alhawiti, 2014; Alqahtani et al., 2023; Shaik et al., 2022). These models apply advanced computational techniques to interpret, generate and manipulate human language in ways that can have a profound impact on educational practices (Younis et al., 2023). Specifically, such technologies enable the automation of complex processes such as student assessment and learning personalization, improving the overall efficiency of teaching and learning (Pardos and Heffernan, 2010).
A fundamental application of NLP models in higher education is in automated assessment and feedback systems (Shaik et al., 2022; Shermis, 2013): NLP models can quickly and accurately analyse students’ work, providing preliminary assessments and detailed feedback, not only speeding up the evaluation process but also providing students with immediate feedback, essential for their learning (Kastrati et al., 2021).
Another significant application is in personalized learning experiences (Xie et al., 2019): using language analysis, these platforms can assess students' proficiency levels and tailor learning content accordingly (Denny et al., 2015; Mathew et al., 2021a). This means that students receive materials and assignments that are optimized for their proficiency level, keeping them engaged and motivated in the learning process (Ni’mah Afif et al., 2020).
Furthermore, NLP models enhance the accessibility of educational content (Mathew et al., 2021b). Through text-to-speech and speech-to-text technology, students with visual impairments or writing difficulties can access educational materials more efficiently (Kudliskis, 2014). This contributes to ensuring that all students, regardless of their abilities, have equal access to education.
So, through the use of these tools, the possibility of personalizing learning aimed at satisfying the specific needs of each category of students opens up (Cox et al., 2019; S. Kumar, 2019). The opportunity to receive a completely new and unique educational approach, tailored to individual needs, constitutes an appreciable value for each student, depending on their abilities and personal interests, helping with homework and providing feedback (Lund and Wang, 2023).
The existing literature on the role of NLP models in education is still in its early stages, with limited studies despite the recent introduction of these technologies (Lim et al., 2023). Preliminary studies indicate growing optimism towards the adoption of tools such as ChatGPT, which is perceived as an enhancer of motivation and engagement for self-taught students. Firat (2023) highlighted how ChatGPT could revolutionize education by democratizing access to knowledge and supporting autonomous learning through interactive dialogue and the ability to explore various perspectives. Despite this optimism, studies like that of Shoufan (2023) also acknowledge challenges, such as the variable accuracy of responses provided by NLP models.
However, there is a clear gap in the literature, specifically regarding the level of student interest in these models and their perception of their added value in education. While technology promises to address challenges such as information overload and educational costs, it is not yet clear to what extent students are truly interested in fully integrating such tools into their learning paths. Consequently, this research aims to investigate these dimensions more thoroughly, seeking to fill the identified gap and better understand the impact of NLP models in the educational context.
Perceived value and intention to use
A useful contribution to understanding the factors that determine a positive attitude of students towards the acceptance and use of NLP models for educational objectives comes from Tiwari et al. (2023). The authors’ study extends the Technology Acceptance Model (TAM) of Davis et al. (1989), which is a widely recognized theoretical framework that clarifies how users come to accept and use new technologies.
TAM has proven applicable to the most disparate contexts and different technologies, such as websites, mobile apps, social media and e-commerce platforms, demonstrating its validity and reliability. Through TAM, some authors have highlighted how the perception of the usefulness of a technology and its ease of use are positively correlated with the intention to use it and this has proven to be applicable to the most disparate contexts and to different technologies (Granić and Marangunić, 2019; Saadé and Bahli, 2005; Salloum et al., 2019).
The literature has also extensively investigated the intention to use new technologies like the degree of willingness of an individual to use innovative tools (Davis et al., 1989; Venkatesh et al., 2012). It represents the complexity that individuals face in performing a certain behaviour, thus offering an indication of the level of effort they are willing or ready to invest in carrying out that behaviour (Al-Maroof et al., 2020; Sheeran, 2002). This perspective underlines the importance of understanding the variety of elements that can influence the intention to use, emphasizing the consideration not only of the will, but also of the efforts involved in the adoption of specific technological behaviours, i.e. the perceived value of NLP models.
Perceived value emerges as a relevant element that affects consumers' decisions, facilitating or preventing the achievement of their objectives through the perceived preference and evaluation of a particular product (Gijón et al., 2023; Woodruff, 1997). In essence, consumers’ different perceptions of value influence their purchasing or intention to use (Abu-Taieh et al., 2022).
The scientific community has often used the VAM, the value-based adoption model (Kim et al., 2007). It incorporates perceived value as a mediating element in the individual decision-making process. Unlike the TAM, which focuses on exploring the intention to use a technology based on factors of usefulness and ease of use, the VAM identified benefits (usefulness and enjoyment) and sacrifices (technicity) as the main determinants of perceived value (Kim et al., 2007).
Holbrook (1999) formulated a categorization of perceived value that includes eight dimensions: convenience, quality, success, reputation, enjoyment, beauty, virtue and faith. Therefore, in general terms, we can state that the perceived value is given by the subjective evaluation of the compromise between the benefits deriving from the service and the sacrifices incurred to obtain it (Aulia et al., 2016; Ogunmola and Das, 2024; Zeithaml, 1988).
Perceived value has provided an essential contribution to understanding consumer behaviour and consumer decisions (Li and Shang, 2020). In studies in the field of information systems, perceived value has been used to explain users' intention to adopt technologies (Turel et al., 2007). Some authors point out that users' belief regarding the perceived usefulness of chatbots is crucial and intrinsic motivation has a positive impact on their intention to use (Brachten et al., 2021) Although perceived value proves to be an effective indicator of technology adoption behaviour, it has rarely been used to examine emerging technologies, such as NLP models. Therefore, we assume that:
Perceived value positively influences university students’ intention to use NLP models.
Gender role
Previous research has highlighted that purchasing decisions, approval, acceptance and implementation of new technologies are influenced by personal factors such as age, gender and education level (Hassan et al., 2010; Venkatesh et al., 2012); among these variables, gender emerges as a key element in internet use (Jackson, 2008).
Although much previous research suggested that males were more likely to adopt new technologies (Breakwell et al., 1986), subsequent literature has indicated a changing trend, with an increasing number of women using technology reducing the impact of gender differences in patterns of technology adoption (Dabholkar, 1992); recent empirical research confirms this perspective (S. Kim et al., 2016).
Venkatesh and Morris, (2000) suggested that men attach greater importance than women to the perceived value of a new technology, being guided in their usage decisions more by instrumental and productivity factors.
Furthermore, the study by Yilmaz et al. (2023) focuses on university students' perceptions of ChatGPT. The authors highlight a positive perception of the tool, but note a significant disparity between males and females regarding the perception of ease of use.
Globally, the population is slowly approaching gender parity: the gender parity score, which measures the proportion of women using the internet divided by the proportion of men using it, stands at 0.92 in 2023. According to ITU (2023), parity will be achieved when we have a score between 0.98 and 1.02. The results obtained regarding gender disparity are lukewarm to date, despite considerable global attention, as also recognized by the Global Gender Gap Report 2023 by World Economic Forum (2023) making renewed action even more urgent. Technological literacy reached 43.7% parity and AI and big data 33.7%. These are among the top 10 skills expected to grow in the coming years, but parity is below 50%; here too, progress is very slow (World Economic Forum, 2023). Yet it turns out that in all skill categories, the gender gap tends to narrow as skill levels increase. In fact, when women are involved, they tend to reach desired skill levels more quickly than men. Cognitive skills that will be crucial in the coming years include creative thinking, analytical thinking and systemic thinking, which stand at 64.3%, 52.7% and 55.6%, respectively, showing even a decline compared to parity levels recorded in 2015 (ITU, 2023). It turns out that accessible online learning on various platforms has characteristics of flexibility and adaptability to users' different needs, but men and women still do not have equal access to these opportunities.
Over the years, efforts to bridge the gap have not been lacking, but they are not enough for a leap forward: more than two-thirds of organizations surveyed for the World Economic Forum’s, 2023 Future of Jobs have implemented Diversity, Equity and Inclusion programs. Yet, despite the proliferation of positive initiatives regarding disparity, as is the case in some particularly active countries, including Italy, the problem persists.
Also, regarding the composition of OpenAI’s audience (Similarweb, 2024), we must note that the distribution favours males: 54.43% versus 45.57% for females (Council of Europe, 2023). But, the EU 2023 report (European Parliament, 2023) pointed out that in Europe, the use of social media is now slightly higher among women than men, regardless of age group. The most popular social networks among women are, in order, Facebook, Instagram, Snapchat, TikTok, WhatsApp and Pinterest, whereas Twitter, LinkedIn, Reddit, Discord, Twitch and Telegram are more prevalent among men. Gender discriminates the preference for social media use: males are more inclined to use them to express opinions, whereas positive social relationships are important to females, who resort to social media to strengthen social ties (Statista, 2023). Furthermore, considering the ranking of the most visited websites worldwide, at the top is google.com: the most visited site in the world in March 2024. In second place is youtube.com, in third place is facebook.com, followed by Instagram.com in fourth place. In tenth place is whatsapp.com, and only in 20th place is openai.com.
In summary, it emerges that the use of technology is only partly related to gender, and while the disparity has slightly decreased over the years, the use of each technological tool represents a separate reality and shows different behaviour concerning gender. But Chat GPT is a relatively recent tool, and because it is not as widespread as the others mentioned, it reflects a behaviour rooted in time: the broader use by males (65.68% versus 34.32% for females) (ITU, 2023), recording a technological disparity rooted in the cultural differences that guide the use and perception of technology.
So, to deepen our understanding of the moderating effect of gender on NLP models, we intend to test the following hypothesis:
Gender moderates the relationship between perceived value and university students’ intention to use NLP models.
Methodology
This study aims to analyse how university students’ intention to use NLP models is influenced by perceived value and how gender is a moderator in this relationship. Using the VAM and context-specific constructs, namely, perceived value and intention to adopt NLP models, we present the proposed research design (Figure 1).
For this research, the VAM by Kim et al. (2007) is used as it is suitable for examining the adoption of NLP models among university students because it focuses on perceived value, analysing how it influences technological adoption. This model easily adapts to specific contexts and can include crucial variables such as perceived effectiveness and gender as moderators. It provides an integrated approach that considers emotional and cognitive factors, allowing for a comprehensive understanding of the user's decision-making process (Kim et al., 2007).
Data collection
To explore and validate the hypotheses outlined in the research design, we proceeded with an email survey addressed to students of the University of Salerno from September to November 2023. The selection of the sample probabilistic was performed with the aim of ensuring accurate representativeness, trying to reflect, albeit on a small scale, the different characteristics of the population being studied to facilitate generalization of the results.
The Web survey, developed through Microsoft Forms, was structured into two components: the first section included ten questions aimed at characterizing the sample, including gender, age and place of belonging, whereas the second section included questions relating to two reference constructs present in the research design. The questions were formulated using a five-point Likert scale, where 1 represents “Totally disagree,” 3 equals “Neutral” and 5 indicates “Totally agree.” The Likert scale is a psychometric approach to measuring attitudes, presenting each item as an agreement/disagreement scale divided into five modalities (Likert, 1932). Participants were asked to express their degree of agreement or disagreement with each statement.
Before the definitive distribution of the questionnaires, a pre-test was conducted: a preliminary version of the questionnaire was administered to a small sample of people (31 students) to evaluate the clarity of the questions. The results of the pre-test indicated the need to make changes to the questionnaire to improve its understandability.
Participation in the study was completely voluntary and kept anonymous, respecting the ethical principles of research.
Respondent profile
A total of 562 valid responses were recorded and analysed, excluding 14 respondents who had failed the attention check. This control process proved effective in filtering out participants who did not meet predefined criteria or who displayed poor attention during the interview. This targeted selection guaranteed the reliability and consistency of the data obtained.
In this section, we focused on the analysis and interpretation of the socio-economic data of the participants, carefully following the objectives of the research. Table 1, following a descriptive analysis, offers a representation in frequency, percentage of the interviewees distributed in the different categories. This visualization allows for an in-depth understanding of the demographic characteristics of the sample, thus helping to contextualize and accurately interpret the results that emerge.
Respondents’ demographic characteristics
| Demographic | Value | Frequency | % |
|---|---|---|---|
| Gender | Man | 154 | 37.93 |
| Woman | 251 | 61.82 | |
| Other | 1 | 0.25 | |
| Age | 18–20 | 234 | 57.64 |
| 21–23 | 103 | 25.37 | |
| 24–26 | 42 | 10.34 | |
| 26+ | 27 | 6.65 | |
| Polo | Scientific | 88 | 15.66 |
| Humanistic | 385 | 68.51 | |
| Natural science | 89 | 15.84 | |
| How many NLP models do you know | 0 | 156 | 27.76 |
| 1 | 217 | 38.61 | |
| 2–5 | 174 | 30.96 | |
| 5+ | 15 | 2.67 | |
| Degree course | Agriculture | 12 | 2.14 |
| Archeology and ancient cultures | 6 | 1.07 | |
| Chemistry | 2 | 0.36 | |
| Data science and innovation management | 3 | 0.53 | |
| Disciplines of visual arts, music and entertainment | 15 | 2.67 | |
| Economy | 99 | 17.62 | |
| Physics | 1 | 0.18 | |
| Physiotherapy | 1 | 0.18 | |
| Global studies | 1 | 0.18 | |
| Informatics | 7 | 1.25 | |
| Engineering | 71 | 12.63 | |
| Letters | 3 | 0.53 | |
| Foreign languages and cultures | 1 | 0.18 | |
| Mathematics | 7 | 1.25 | |
| Medicine | 72 | 12.81 | |
| Dentistry and dental prognosis | 8 | 1.42 | |
| Biological science | 3 | 0.53 | |
| Cultural heritage sciences | 1 | 0.18 | |
| Heritage sciences | 1 | 0.18 | |
| Communication sciences | 14 | 2.49 | |
| Education sciences | 172 | 30.60 | |
| Legal sciences | 5 | 0.89 | |
| Nursing sciences | 8 | 1.42 | |
| Motor sciences | 3 | 0.53 | |
| Political science | 6 | 1.07 | |
| Statistics for Big Data | 39 | 6.94 | |
| Diplomatic studies | 1 | 0.18 |
From the analysis of Table 1, a marked prevalence of the female gender compared to the male gender clearly emerges, with a percentage of 67.26% compared to 32.38% of the participants. Equally clear is the predominance of interviewees falling within the age group between 18 and 20 years, which constitutes 58.36% of the sample examined. Furthermore, it is noted that the vast majority of participants, equal to 68.51%, are affiliated with the Humanities pole, whereas the remaining two poles, namely, Scientific and Natural Science, present an almost equal distribution.
Such contextualization provides a solid basis for the interpretation of the collected data and contributes to a more complete understanding of the profile of the research participants.
Construct reliability and validity
To ensure the effective validation of research instruments and scales, it is crucial to adopt a methodologically robust approach in acquiring data from participants. This research used measurements from previous investigations relating to VAM. To ensure adaptation to the needs of this research, the operational definitions of the constructs were carefully analysed, taking into account the current evidence present in the body of scientific literature. This approach allowed a reliable construction of the survey instruments (Table 2), helping to consolidate the validity of the data collection process in accordance with the specific purposes of this study.
Measurement elements and their references
| Construct | Item | Definition | References |
|---|---|---|---|
| Perceived value | PoV_1 | Perceived value is manifested through the individual evaluation of the balance between the benefits offered by a service and any sacrifices or efforts necessary to obtain them | (Aulia et al., 2016; Kim et al., 2007) |
| PoV_2 | |||
| PoV_3 | |||
| PoV_4 | |||
| PoV_5 | |||
| Student’s intention to use NLP | IoS_1 | It measures the intensity of the attitude towards the intention to perform a particular behaviour | (Choi and Drumwright, 2021; Shen et al., 2022) |
| IoS_2 | |||
| IoS_3 | |||
| IoS_4 | |||
| IoS_5 |
In order to conduct a comprehensive assessment of the validity and reliability of the instrument used in the research, a confirmatory factor analysis was performed. This methodological approach aims to explore the factor structure and confirm the validity of the adopted measures. At the same time, to examine the internal consistency and convergent validity of the indicators, three important indicators were calculated: Cronbach’s α, which provides a measure of the internal consistency between the items; the average variance extracted (AVE), which indicates how much the constructs vary with respect to the error measurements, and the composite reliability (CR) which represents the overall precision of the measurements. This in-depth statistical analysis ensured robust validity and reliability of the measures used in the study.
With reference to Table 3, it is possible to observe that both Cronbach’s alpha values exceed 0.7, exceeding the threshold of 0.9, thus indicating a high reliability of the measurement scales (Liu et al., 2013). All factor loading coefficients of the items are higher than 0.7. The Analysis of Extracted Variance (AVE) for the constructs highlights satisfactory values (AVE_PoV 0.695; AVE_IoU 0.753), whereas the Composite Reliability coefficients (CR) exceed 0.8. Consequently, it can be stated that the convergent validity of the measurement scales is robust (Liu et al., 2013). These results attest to a consolidated internal consistency and adequate convergence of the measures, providing a reliable basis for the use of the rating scales in the context of this research.
The models to test the hypotheses
In order to test our first hypotheses we have estimated the following OLS model:
where the dependent variable is measured using the construct Intentions of Use that we derived from the questionnaire administered to the students of the University of Salerno. On the right-hand side of the equation, the Perceived Value of NP technologies is the independent variable of interest, that we derived from our questionnaire. We also included a set of variables to control spurious effects. Specifically, we considered the gender of the respondents, the age and the field of study where they are involved in. The field of study was operationalized through a dummy variables for the three area of knowledge used by the European Research Council, namely, Life Science (LS), Social Sciences and Humanities (SH) and Physical Sciences and Engineering (PE). In the model, we included the dummies for LS and SH. The PE field is the reference category.
To test the second hypothesis, which concerns the moderating effect of gender on the relationship between perceived value and intentions of use of NP technologies, we added to the model (1) the two-way interaction term between the gender dummy and the variable perceived value follows:
The models are estimated with robust standard errors.
Results
In this section, we present the results of the models used to test our hypotheses. Table 4 shows the results of the regression models, using robust errors. The coefficients reported in Columns 1 and 2 indicate a significant positive association between the intentions to use NLP models linked to AI and the perceived value of these technologies (β = 0.797, p > 0.1%). Furthermore, it emerges that the gender of the interviewees influences this relationship. In Column 2, the interaction term between perceived value and gender of the respondent is found to be negative and statistically significant (β = −0.121, p > 5%). This suggests that the connection between perceived value and intentions to use NLP models presents significant variations between males and females. Therefore, the second hypothesis of the present study, which hypothesized a different slope in the relationship between perceived value and intentions to use AI technologies based on gender, cannot be rejected due to the presence of sufficient evidence.
Regression results of the relationship between perceived value and intention of use of AI technologies
| Intentions of use | Intentions of use | |
|---|---|---|
| 1 | 2 | |
| Perceived value | 0.797*** (25.94) | 0.872*** (23.22) |
| Gender dummy | −0.0735 (−1.06) | −0.0712 (−1.04) |
| Age | −0.00533 (−0.69) | −0.00575 (−0.74) |
| Social science and humanities (SH) | 0.213** (2.70) | 0.218** (2.77) |
| Life science (LS) | 0.100 (0.88) | 0.0976 (0.87) |
| Gender × Perceived value | −0.121* (−2.09) | |
| Constant | 0.0207 (0.12) | 0.0217 (0.13) |
| N | 404 | 404 |
| F | 163.6 | 160.3 |
| R2 | 0.640 | 0.644 |
*, ** and *** denote significance at the 5, 1 and 0.1% levels, respectively
Furthermore, in Figure 2 the correlation between perceived value and intentions to use NLP models has been outlined, distinguishing between males and females. From the visual analysis, the positive inclinations of the relationship between perceived value and intentions of use clearly emerge for both males and females. In other words, for both male and female genders, the increase in perceived value is associated with an increase in intentions to use AI technologies. However, it is important to note that the line representing males' usage intentions has a significantly greater slope than that corresponding to females. This suggests that, although there is a positive correlation between perceived value and usage intentions in both genders, this association is more pronounced among males.
The line graph shows intentions of use on the vertical axis ranging from minus 1 to 1.5 and perceived value on the horizontal axis with two categories: Low Perceived Value and High Perceived Value. Two lines represent Male and Female groups. For the Male group, intentions of use are about minus 0.8 at Low Perceived Value and about 0.9 at High Perceived Value. For the Female group, intentions of use are about minus 0.8 at Low Perceived Value and about 0.7 at High Perceived Value.Interaction effects of gender dummy on the relationship between perceived value and intentions of use of AI technologies
The line graph shows intentions of use on the vertical axis ranging from minus 1 to 1.5 and perceived value on the horizontal axis with two categories: Low Perceived Value and High Perceived Value. Two lines represent Male and Female groups. For the Male group, intentions of use are about minus 0.8 at Low Perceived Value and about 0.9 at High Perceived Value. For the Female group, intentions of use are about minus 0.8 at Low Perceived Value and about 0.7 at High Perceived Value.Interaction effects of gender dummy on the relationship between perceived value and intentions of use of AI technologies
Table 5 shows the results of the simple slope test for the two lines drawn in Figure 2. For both genders, male (β = 0.872, p < 0.1%) and female (β = 0.751, p < 0.1%), the slope of the relationship of our interest is positive and highly significant. It is interesting to note that, although a low perceived value does not show significant differences in intentions of use between males and females, in the presence of a high level of perceived value, intentions to use AI technologies in males are significantly higher than those manifested by women (β = 0.192, p < 5%).
Therefore, we conclude that there is not sufficient evidence to reject our two research hypotheses (Table 6).
Result's hypothesis
| Hypothesis | Result |
|---|---|
| Hp 1 | Not rejected |
| Hp 2 | Not rejected |
To further investigate our results we have examined the impact of the areas of knowledge in which the students are involved in by investigating if the moderating role of gender on the relationship between perceived value and intention of use on NP technologies depends on the fields of study of the students interviewed.
Therefore, we have re-estimated the model in Table 4 by introducing a three-way interaction considering the gender and the fields of study of the interviewees as moderators of the relationship between perceived value and intentions of use of NLP technologies. In Table 7, we have reported the results of the OLS model with robust standard errors estimating the jointly moderating effect of the gender and the fields of study.
Results of the OLS model testing the three-way interaction between gender, the fields of study and the perceived value of the NPL technologies
| Intentions of use | |
|---|---|
| Perceived value | 0.882*** (10.55) |
| Gender dummy | −0.200 (−1.56) |
| Age | −0.006 (−0.80) |
| Social science and humanities (SH) | 0.077 (0.71) |
| Life science (LS) | 0.077 (0.65) |
| Gender × Perceived value | −0.268* (−2.02) |
| Social science and humanities (SH) × Perceived value | −0.038 (−0.40) |
| Life science (LS) × Perceived value | −0.004 (−0.04) |
| Social science and humanities (SH) × Gender | 0.245 (1.60) |
| Life science (LS) × Gender | 0.020 (0.09) |
| Social science and humanities (SH) × Gender × Perceived value | 0.261† (1.79) |
| Life science (LS) × Gender × Perceived value | −0.088 (−0.38) |
| Constant | 0.089 (0.51) |
| N | 404 |
| F | 102.17*** |
| R2 | 0.66 |
†, * and ***denote significance at the 10, 5 and 0.1% levels, respectively
The model is significant (F = 102.17, p < 0.1%) and the results highlight that the three-way interaction term is significant (β = 0.261, p < 10%). Therefore, the slope of the relationship between perceived value and intentions of use of NP significantly depends on the gender of the students and on the fields of study.
For a more detailed investigation on the existence of a jointly moderating effect of the gender and the fields of study on the relationship of our interest, we estimated the slope of such a relationship for males and females and for the different fields of study. In Figure 3, the relationships between perceived value and the intentions of use of NLP technologies for male and female students and for the three fields of study investigated are plotted.
Three line graphs are titled Physical Sciences and Engineering, Social Sciences and Humanities, and Life Science. The vertical axis shows intentions of use ranging from minus 1 to 1. The horizontal axis shows perceived value with two categories: Low and High. Each graph includes two lines representing Males and Females. In Physical Sciences and Engineering, intentions of use increase from about minus 0.7 to about 0.8 for Males and from about minus 0.8 to about 0.4 for Females. In Social Sciences and Humanities, intentions of use increase from about minus 0.6 to about 0.9 for both Males and Females. In Life Science, intentions of use increase from about minus 0.8 to about 0.9 for Males and from about minus 0.6 to about 0.4 for Females.Plot of the intentions of use of NPL by perceived value (low vs high), gender (males vs females) and fields of study
Three line graphs are titled Physical Sciences and Engineering, Social Sciences and Humanities, and Life Science. The vertical axis shows intentions of use ranging from minus 1 to 1. The horizontal axis shows perceived value with two categories: Low and High. Each graph includes two lines representing Males and Females. In Physical Sciences and Engineering, intentions of use increase from about minus 0.7 to about 0.8 for Males and from about minus 0.8 to about 0.4 for Females. In Social Sciences and Humanities, intentions of use increase from about minus 0.6 to about 0.9 for both Males and Females. In Life Science, intentions of use increase from about minus 0.8 to about 0.9 for Males and from about minus 0.6 to about 0.4 for Females.Plot of the intentions of use of NPL by perceived value (low vs high), gender (males vs females) and fields of study
Figure 3 highlights that the lines are positively sloped for male and female. However, while for the field of study Social Science and Humanities the two lines do not differ significantly, for Physical Sciences and Engineering and Life Science the relationship between perceived value and intentions of use of NLP technologies is more sloped for males than females. Table 8 presents the difference slope tests for the lines plotted in Figure 3.
The results show that for the students in the fields of study pertaining to PE (β = 0.268, p < 5%) and LF (β = 0.355, p < 5%) the relationship between perceived value and intentions of use of NLP technologies is more sloped for males than females. In contrast in the field of Social Sciences and Humanities the slope of such a relationship is not significantly different for males and females (β = −0.041, p > 10%). A comprehensive interpretation of these results suggests that in the areas of knowledge related to PE and LS males and females differ in their propensity to the use of NLP technologies, specifically males demonstrate a greater propensity to the use of NLP technologies as the perceived value increases. In contrast, in the field of Social Sciences and Humanities, the intentions for using NLP technologies are the same for male and female interviewees.
Discussion
Research findings regarding the perceived value and intention to use NLP models confirm what the existing literature emphasizes regarding new technologies. There is a positive and recognized correlation between two theoretical constructs, which can also be found in our data. University students recognize the value of these innovative tools and this is a prerequisite for their use.
The relevance of our research lies in the intention to use the gender variable as a moderator of the relationship between perceived value, through the VAM and the intention to use NLP models. These tools based on AI are gaining great popularity and many authors are underlining the fundamental role they can play in all phases of school and academic education (George and Wooden, 2023).
From a pragmatic point of view, the results emerging from the research suggest that the integration of NLP models into university training programs is not only possible but could constitute a key element (George and Wooden, 2023). This integration would contribute not only to enriching the students' learning experience but also to guarantee cutting-edge training in line with the needs of the digital society (Malik et al., 2023).
The results highlight that the relationship between value perception and intention to use can vary noticeably for men and women. Consistent with previous educational research studies examining gender differences in the perceived usefulness of technologies, the present study found that male students placed greater value on technological tools than female students (Koohang, 1989; Shashaani and Khalili, 2001). Venkatesh and Morris (2000) suggested that men attach greater importance than women to the perceived value of a new technology, being guided in their usage decisions more by instrumental and productivity factors. In this wake, the present work confirms these findings and adds a critical dimension to the theory on technological adoption, with respect to the use of NLP tools.
Several studies highlight the inadequacy of students' development of digital skills, especially advanced ones, together with an alarming gender gap. Women reveal that they mostly consider their digital skills to be low or poor, and to a much greater extent than men (della Volpe et al., 2024; ITU, 2020).
This in-depth analysis contributes to a more complete understanding of the gender role in the dynamics related to the adoption of advanced technologies, such as NLP models. The theoretical implications not only extend the understanding of the behavioural dynamics associated with the adoption of NLP models but also provide a solid basis for designing more effective education strategies. In synthesis, sexual gender difference may be appropriate to make the adoption of such models more effective.
Even from a pragmatic point of view, the results emerging from the research suggest that university students are ready and predisposed to integrating NLP models into education and training programs.
Implications
The implications are diverse and affect various aspects of the academic environment.
To ensure students make correct and appropriate use of NLP models, teachers could introduce practical demonstrations of their potential, such as automatic draft correction or assistance in information retrieval, to concretely illustrate their advantages. For example, they can be used to develop personalized virtual assistants for study purposes or to enhance writing skills by providing real-time feedback on grammar and style. Programs like Grammarly or ChatGPT can analyse written texts and suggest improvements, aiding students in developing a clearer and more persuasive writing style. In addition, educational chatbots can act as virtual tutors, providing responses and explanations on complex topics at any time, as seen in platforms like Duolingo or IBM Watson.
Furthermore, the diversity in the usage of such models between genders underscores the importance of developing tailored training strategies. Educators should propose programs aimed not only at bolstering the confidence of all female students in NLP models and AI technologies but also at reducing gender disparities in their adoption. Specific workshops and seminars could be organized to further engage female students, thus encouraging greater participation and involvement.
These elements have positive effects on management strategies, opening up opportunities in both the academic and business sectors for the development of innovative training methodologies. Such methodologies can be specifically designed to overcome barriers associated with the gender gap in the technology sector. Furthermore, policies can be developed to ensure fair access and effective use of NLP technologies. The adoption of these strategies would not only promote greater inclusion and female participation in STEM fields but could also enhance organizational competitiveness and innovation, key elements for growth and success in both the academic and corporate contexts. Today, it is essential to invest in continuous training and professional development programmes to keep staff up-to-date on the latest technologies and best practices. This aspect is particularly crucial in technologically advanced sectors, where professional skills can enhance career prospects and positively influence the perception of technology use across different gender identities.
Moreover, in an era where the “third mission” becomes a cornerstone for universities, it is strategic to establish collaborations among academic institutions, training entities and businesses to develop integrated training programmes, share best practices and resources and cooperate in research projects that explore new applications of NLP to promote gender equity.
Limits and further research
Our research makes a significant contribution to understanding the use of NLP models in academia, but has some limitations that require further investigation. First of all, the generalizability of the results is limited by the selection of the sample, composed exclusively of students of the University of Salerno. This geographical and cultural specificity may not fully reflect the variety of perspectives and behaviours present in other university contexts, both national and international.
Secondly, the research focuses solely on NLP models, neglecting to explore how other AI technologies might influence students’ perceptions and intentions. Integrating different AI technologies could enable a deeper and more nuanced understanding of technological dynamics in university education.
In addition, given the importance of the perceived value as a determinant in the intention of university students to adopt NLP models, it is suggested that future research include an analysis of the degree of familiarity of teachers with such models and the frequency with which they integrate and propose them in educational programs; this could also involve an evaluation of teachers’ specific skills in using NLP tools and their pedagogical practices to facilitate student learning. Special attention could be devoted to identifying any barriers or challenges encountered by teachers in effectively incorporating these models into teaching.
Finally, further exploration of gender disparities in relation to different academic areas is hoped for, to delve into how gender differences influence the adoption and use of NLP models among students from various fields of study, as well as to examine any strategies or interventions that could be implemented to develop more inclusive and equitable approaches.
Conclusions
The study examined the positive relationship between the perceived value of NLP tools in the university environment and their intention to use them. The results indicated that NLP models are now widely known and recognized as valuable resources that support students in their academic activities. The positive correlation between perceived value and usage intentions constitutes a robust theoretical foundation for understanding students’ motivations for embracing new technologies (Montalvo et al., 2018; Visvizi et al., 2021).
Furthermore, the data highlights that the sexual gender variable influences the aforementioned correlation differently and that men therefore have a greater predisposition to using these tools.
The analysis of the results of this research emerges as a significant reflection for the academic context in the current era: in an environment capable of recognizing the value of new technological tools, it is necessary to respond with training methodologies that contemplate these models, crucial support in the processes of learning aimed at new generations (Ozdamli and Cavus, 2021; Younis et al., 2023). The use of NLP models can improve the accessibility of education for students with special needs, allowing text to be converted into spoken language to assist those with reading or hearing difficulties. However, this requires a balanced approach to avoid unintended side effects. Student training on the conscious use of NLP is therefore essential to maximize its benefits while minimizing unintended side effects, ensuring that this technology serves as an ally in education.
According to Deloitte (2023), generative AI is in its early days, whereas the technology goes fast. Embracing Generative AI means changing our vision of the future and assessing risks and benefits in all sectors, from education to business, from health care to finance. Governance and risk mitigation are needed to grasp the new opportunities offered (Boninsegni et al., 2022). As we know, the ChatGPT tool can create output similar to that generated by human beings through their creativity, thought and effort in different ways (text, format, audio, image, video, 3D/specialized), and each one can create value. Generative AI is much more than a simple chatbot (Bahroun et al., 2023): it can unleash innovation, transform the educational way, amplify new systems and, under the impetus of the benefits to be gained, could act, if well used, as a balancing instrument in the gender gap. Using an intuitive interface to be easily interrogated, could develop and enhance digital skills. Curiosity and strong motivation on the part of female students could offset perplexities towards this tool and trigger a new path towards increased digitization.
Understanding in even more detail the differences that men and women show in the adoption of NLP tools would allow for more effective creation of educational and learning models personalized to different needs. Future studies could investigate in greater detail the components that define perceived value so as to trace further advances in the management literature and provide even more operational ideas for practice.
