Editorial: Taking care of the individual: well-being in the age of algorithms

The rapid advancement of artificial intelligence (AI) technologies is significantly impacting individuals' lives. AI systems, from recommendation algorithms and virtual assistants to autonomous vehicles, large language models and AI agents, are reshaping the way people think, live, work and interact with the world (Liang et al., 2021). The emergence of generative AI, such as ChatGPT and other similar tools, has raised new opportunities and challenges for individuals. These technologies impact how individuals create, consume and interact with information (Henkenjohann and Trenz, 2024). They offer novel ways to not only increase creativity and productivity but also raise important ethical and societal concerns (Rahwan et al., 2019) and can impact the well-being of individuals.

AI has swiftly become an integral part of information systems (IS) research, advancing our understanding of its consequences for individuals, organizations and society. This workshop focuses on the interactions between AI and the individual. With technologies' positive as well as negative impacts on individual well-being, designing IS for well-being has become increasingly important (Spiekermann et al., 2022). Therefore, this workshop aims to examine the AI world from the individual's perspective (Matt et al., 2019). It consolidates diverse perspectives on AI's influence on the individual and invites interested researchers to examine how AI shapes personal and social experiences.

Research in this area is crucial for understanding the implications, opportunities and risks associated with AI-driven interactions. It can help developers create more user-centric AI technologies and services that enhance individual experiences and well-being. Additionally, it can inform companies about implementing AI tools in ways that align with user values, are ethical and promote overall well-being. To summarize, focusing on AI's influence on the individual, this workshop seeks to foster a comprehensive understanding of human interactions with AI and its consequences.

To explore and address this timely and impactful topic, we organized a panel discussion with four experts. Our objective was to discuss and evolve viewpoints on AI in individuals' lives. This discussion was held as part of the Digitization of the Individual (DOTI) workshop titled “The Digitized Individual – Artificial Intelligence and the Individual” in Bangkok, Thailand, which was held in conjunction with the International Conference on Information Systems in 2024. The four panelists in alphabetical order included Alexander Benlian, Darmstadt University of Technology, Germany; Kevin Bauer, Goethe University Frankfurt, Germany; Sirkka Jarvenpaa, The University of Texas at Austin, USA, and Wai Fong Boh, Nanyang Technological University, Singapore. The panel was moderated by Hamed Qahri-Saremi from Colorado State University and Ofir Turel from the University of Melbourne.

The panelists' research experiences with AI in individual lives were diverse, reflecting different motivations and areas of focus. For example, Wai Fong is currently examining how we can apply traditional team theories to understand human-AI teams and how we can use GenAI in ways that preserve individuals' critical thinking. Kevin is currently conducting extensive research on human-AI interaction, with a particular focus on the well-being of the parties affected. Sirkka views herself as both a behavioral and organizational researcher, particularly interested in emerging technology topics. Alexander's research also centers on human-AI collaboration and well-being. Their detailed conversations are summarized here:

Wai Fong: We now see that AI is very ubiquitous, and it's affecting all of us. When we think about well-being from an individual perspective, we can categorize its impact based on our different roles. First of all, as consumers interacting with the technology, clearly there are positive and negative impacts. On the positive side, it's helping us to improve learning, creativity and productivity. The negative side is also obvious. As a parent, I worry about how it is impacting my daughter in terms of her well-being, the cybersecurity risks and the impact on her mental health, growth and development. We should also probably be thinking about our role as employees. There's already some work about the impact of algorithms on individual employees and how AI can be used to control and optimize their time. For example, algorithms can have serious psychological impacts for gig workers, on top of their positive impacts.

The other thing we should be thinking about is the impact on the non-users. We should be thinking about the idea of the digital divide. Non-users – those who do not have access to the technologies – are impacted by their inability to use them. On the flip side, we should think about designing some of these algorithms; we all know about the biases and the possible negative impacts. Another aspect that is often neglected is the potential negative consequences if the AI itself is not designed well and ethically and does not consider human centricity. We also need to think about the implicit implications for individuals who might not be users per se, but they are impacted because of other people's use of these algorithms. These could be individuals who are not selected for jobs because of the way that the AI algorithms are designed in a biased manner or people who might be disconnected because of the technology that is designed in ways that exclude them. In addition to the direct impact on the users, we should also be considering the non-users, those people who did not choose to use the technology but are inadvertently affected by these technologies.

Kevin: AI is a general-purpose technology, and we basically want it to improve our lives, making processes more efficient. When humans and AI collaborate efficiently and well, a key factor is that the individual who engages with the AI actually feels good about it and has some well-being improved by using AI; otherwise, there is hardly a purpose for that, right? If the mental health of a person who collaborates with AI is dramatically impaired, then is there actually a benefit of AI? I would seriously question that. From this perspective, there might be a very interesting relationship between people's well-being and the capability of realizing and leveraging complementarities, which can help to move societies forward and to make processes better and more efficient and, ultimately, change the way people use them. It's really in our best interest to understand how this technology affects people's well-being physically and mentally, as well as from the perspective of our capabilities to continue to learn. If we rely too much on these technologies and if we have a technology at hand that does a lot of the cognitive and effortful work for us, do we risk losing our ability to think critically and just take everything for granted? From an aggregate-level perspective, if it harms individuals, then the entire society may converge to a new equilibrium. It should always be this bottom-up approach where we look at the individual or how people work with this technology, how they leverage it, how they feel about it and whether it makes individual lives better. Thereby, we can make sure that we don't lose some of the people along the way and exacerbate the huge divide and biases that already exist in our society. Therefore, we should pay more attention to how people's well-being is affected on an individual level when they interact with this technology. We should also understand how well-being affects the way that this technology can help our societies.

Sirkka: All right, we've talked about individuals and society, so I will focus on the meso level – the organizations. When I first saw the question, “Why should we care about well-being in the age of algorithms?”, my initial thought was, “Haven't we always cared? What's really different today?” That led me to reflect on my research, particularly on organizations dedicated to protecting those who are silenced. This includes political prisoners but also juveniles who are sentenced as adults, which is quite common in the USA. I would say that digitization, such as predictive algorithms, has a dramatic impact on longstanding organizations – such as Amnesty, Red Cross and Greenpeace – whose mission has been centered on the well-being of citizens, particularly vulnerable citizens. What are we seeing today? Individual well-being cannot be disconnected from these critical institutions that society needs. How are we supporting? How can technology further strengthen those critical organizations in a society?

Alexander: In my view, the need to care about individual well-being when interacting with algorithmic systems stems from the very nature of these algorithms. We know they are far from neutral, often designed to serve commercial or power-driven interests rather than individual flourishing. As IS researchers, we have a responsibility to scrutinize how such systems are embedded in everyday life and how they affect people on a personal level. We've already seen that algorithms can deceive, exclude or trap individuals in filter bubbles. These developments are no longer marginal – they are penetrating core areas of human intelligence and cognition. This represents a qualitative shift and demands that we focus even more on the implications for well-being. Encouragingly, this also opens up rich opportunities for DOTI research to explore new conceptual and empirical dimensions of well-being in the age of AI.

Kevin: I just wanted to add to the discussion about individuals and the potential problems that they may have with this technology. Looking at history, when we do not care about individual well-being – especially in the first waves of industrialization – it has led to huge political and economic crises and upswings. There is a historical interpretation that communism, Nazism and other political crises originate from large cultural and societal shifts, often driven by technological advancements. People needed to find new vacancies as their old jobs were just replaced by technology. Many people lost their jobs, and that led to huge societal problems because those who introduced technology did not look at the individual. They were ignored; they were left out. A big problem with AI is that we should focus more on the fact that, in the past, technological developments took a long time and gradually adapted to new technologies. But now, all of these developments are so fast. Three years ago, no one talked about ChatGPT, and now everyone is talking about it. It is a technology that is everywhere. In this rapid development, if we forget to think about individuals and leave them behind, they have even less time to adapt than before, which could lead to even greater societal crises. People need to find their places. They want to feel well, and they want to have good well-being. They want to have a stance in life. Therefore, I think the focus on the individual is not only crucial at the personal level but also a core issue we need to face from a higher societal perspective.

Wai Fong: I'd like to approach this question by drawing on what we've learned from past research and theoretical perspectives that can help us study this space. One important lens is the socio-technical perspective – the idea that we must consider both social and technical elements together. Traditionally, IS researchers have focused on either the adoption or the development of technologies. But with AI, we need to integrate these two, because the technology evolves as it's used – it is trained on user data and constantly updated. So, rather than treating use and development as separate, we need to study their interplay. Another key point is the shifting roles in this ecosystem. We have developers and users, of course, but now the technology itself is taking on a more active, evolving role. Some scholars even describe AI systems as having agency – capable of learning and developing independently. That opens up new questions about how these three agents – users, developers and the AI – interact over time. Finally, there's the ecosystem perspective. AI doesn't operate in a vacuum. Like other technologies, it exists within broader social, regulatory and industrial contexts. But with AI, rules and standards are still evolving, and they influence – and are influenced by – the technology's development. So, regulators and industry actors must be considered part of this dynamic ecosystem.

All of these perspectives – the socio-technical, the evolving roles, and the ecosystem view – offer opportunities for IS researchers to apply existing theories in new, nuanced ways to study AI and its implications for emotional, physical and psychological well-being.

Sirkka: My view is that there is no individual without the collective. This shift is less about new knowledge and more about the transformation of identity – how we think about individuals and how we assign rights and responsibilities that were once tied solely to individuals but now potentially to other entities. This raises questions about the changing societal and organizational norms. What really strikes me is the global crisis of loneliness, especially among young people. This is happening at a time when we have incredibly powerful tools to connect in many ways. So where have we gone wrong? How is it that we're experiencing rising loneliness despite increasing connectivity? Metcalfe – you all know Metcalfe's Law – argues that connectivity has had and will continue to have a greater impact than many of the current conversations around AI. So we need to reflect on what connectivity has meant for us and how that insight can help us understand today's transformations – whether we frame them as shifts in identity or in intelligence.

Alexander: I'd like to add a point – we've heard a lot about the meso-level and sociological effects of AI and GenAI, but we shouldn't forget the psychological effects, particularly the erosion of cognition. That's a significant concern when using GenAI: the risk of overreliance and dependency on these tools. Going forward, it's increasingly important to equip users with AI literacy across multiple dimensions. They need critical thinking and problem-solving skills to understand how algorithmic systems work. This should also be reflected in our curricula. As IS researchers, we shouldn't only focus on AI development and adoption. If we're committed to a socio-technical approach, we also need to address the human side – supporting users with the skills and capabilities they need to engage with these technologies responsibly.

Kevin: Maybe one final thought on what's been said. I think the IS discipline is particularly well-suited to address these challenges. Unlike computer science, IS has always focused on the social component, on the individual as part of the system. AI introduces a new kind of externality in individual behavior. We now have tools that learn from how we use them – something no tool in the past could do. A screwdriver, for example, doesn't change based on how you use it. But with AI, your specific behavior can shape how the tool works – not just for you, but for others. You contribute data and information that can influence outputs for future users. As Alexander mentioned, there's already early evidence, particularly with GenAI, that overreliance on these systems could lead to a kind of collapse. When too much synthetic data is generated and fed back into the system, human knowledge and creativity – especially in areas like imagination – could decline. This, in turn, could distort the distribution of available content, degrade model performance and negatively impact future generations of users and systems. So, our behavior today – how we use these systems as individuals – has long-term effects on how others will experience and benefit from them. This should not be underestimated.

Alexander: I think we've already touched on this, but the conversation around user well-being has clearly shifted to more foundational topics, such as identity and self-concept. In the past, the focus was often on less profound aspects – important but not as deep as the questions we're grappling with now. With GenAI, the boundaries of agency, creativity and productivity have shifted more dramatically than before. As a result, research is starting to explore how these shifts impact well-being – not just at a surface level, but in multiple dimensions, including mental well-being, cognition and emotional experience. There's a qualitative change in how we're approaching the topic: we're now asking more philosophical questions about the nature of artificial vs. human intelligence and what that distinction means for users and their well-being.

Kevin: I think GenAI offers huge opportunities but also significant dangers. Studies show it can help level the playing field, especially for those with lower skill levels. But this comes with costs and risks that users need to be aware of. Anyone using GenAI tools – like ChatGPT, for instance – should critically reflect on the output they're receiving. Not long ago, there were no warnings or disclaimers in browser windows telling users to reflect critically on AI-generated content. Now we're seeing things like image labeling and watermarks to indicate GenAI-produced media. This signals a deeper issue. I recently read an article – possibly in the New York Times (NYT) – that described this moment as an era of deep doubt (a term derived from “deep learning”): a fundamental crisis of trust in information. What is true? What can we rely on?

There's really no way around this other than to strengthen individual agency – to give users the tools and responsibility to ask the hard questions: Is this real? Can I rely on this? We need critical thinking and multiple sources, especially since, from a technical standpoint, we still don't fully understand how these systems work. We can't entirely prevent hallucinations or inaccuracies. So it comes back to the individual – they must be equipped with tools and knowledge to navigate this landscape. I fully agree with Alexander: we need new forms of literacy. But agency also has to be supported from higher levels – governments and organizations must provide users with the abilities, tools and technologies that enable this agency and help mitigate the negative impacts of GenAI on our lives.

Alexander: Yes, that's a good question, and I think I have two parts to my answer.

First, on conceptualization: In my view, we've taken a narrow approach to defining well-being in IS research. We've often focused just on positive or negative emotions, or sometimes life satisfaction. But I think we can learn a lot from positive psychology. For example, Martin Seligman's PERMA model includes broader dimensions – such as positive emotions, engagement, relationships, meaning and accomplishment. Incorporating such frameworks could really help us broaden how we define well-being in our work.

Second, on measurement: I think we need to move beyond single-shot surveys or isolated experiments, which are common in IS research. With more data now available – such as trace data, diary studies, or experience sampling – we can ask longitudinal and dynamic questions. For example, how does well-being evolve over time? How do different aspects of well-being develop within users as they interact with AI? So there's a real opportunity to expand both our conceptual and methodological toolkit in this space.

Sirkka: Is the term “well-being” actually meaningful in this context? It's hard to think of anything that wouldn't fall under it. For example, how can you have well-being without basic security? And from there, we move up Maslow's hierarchy, all the way to self-actualization. It's a bit like discussions around purpose-driven organizations. To me, well-being means being able to pursue the purposes that matter to you at a particular moment in time. It's about having the conditions that allow you to follow what's meaningful for you.

Wai Fong: I think this relates to what I mentioned earlier about seeing design and use as mutually influential. This interplay can take various forms, but one key aspect is to embed values early on in the design process. That requires some level of alignment between users and designers. We also previously discussed the role of regulations and standards as important parameters. Anyone who has taught ethics in AI knows there's rarely a clear right or wrong – especially because it's difficult to achieve true human centricity when multiple stakeholders are involved. What benefits one group may negatively impact another. This means we need to acknowledge the potential for controversy upfront and create space for those discussions early in the process. In other words, we must influence ethical design during the development phase, rather than evaluating the consequences only after deployment. All of these tie back to the idea of integrating the social and the technical – which means involving users, developers and the technology itself as interconnected elements. These are some of the perspectives we should keep in mind when thinking about ethically grounded AI design.

Kevin: Right now, there's a mismatch between the speed of technological development – especially in AI – and the ability of governments to create responsive frameworks. Take the EU AI Act, for example: the debate began as early as 2013, and just as it was nearing adoption, GenAI emerged, throwing things into crisis mode and prompting another six months of debate to account for these new developments. Too often, policymakers react quickly without fully understanding the downstream consequences of regulation. We've seen this before with GDPR. It created a lot of uncertainty – organizations weren't sure what data they could store or how to proceed, leading to a kind of regulatory paralysis. I'm now seeing similar hesitation around GenAI adoption.

As IS researchers, we're uniquely positioned to understand the interplay between technology, individuals and organizational structures and to identify unintended consequences of regulation. In one of my projects on transparency, for example, we found that mandating transparency doesn't always lead to better outcomes. Disclosure alone doesn't solve every problem – “sunlight is not always the best disinfectant.” Our role can be to help policymakers understand user responses and broader ramifications so they can gradually improve regulation.

Alexander: I'd like to emphasize something: as researchers, our role does not stop at doing the research. We also have an obligation to inform. That means participating in standards-setting bodies and policy committees – playing an active, normative role in society by injecting our knowledge directly into regulatory processes. This is part of our responsibility, too.

Wai Fong: And beyond just policies that shape technological development, we need to think more broadly – especially about the implications for end users. What happens to people whose skills become obsolete due to AI? What policies do we need to ensure that displaced individuals can reskill, upskill and maintain a sense of identity and purpose? We must help policymakers think through not just the technical impact but also broader societal effects: How do we support people whose occupations are disappearing? How do we help them adapt and find new roles? These are key policy implications we can't overlook.

Kevin: From a technical perspective, once the data is in the AI machine, then we have a problem of making it forget that data. From a technical perspective, generative AI models have the fundamental ability to “understand” data, and then we have integrated database systems that they can draw data from. I think maybe we need some fundamental models that have skills of making sense of the structure of data and producing synthetic data to be used for training the AI models, whose structure is similar to the actual data in databases that we can control. Then, users can control the data in the databases, which will be used to generate synthetic training data. A case in point is the NYT vs. OpenAI court case, wherein even showing that ChatGPT can reproduce a proprietary NYT article word by word mathematically does not 100% prove that this data was included in the training data. Because there are just so many combinations of words that can be used, there is a tiny probability that this is just not the case. Therefore, once the data are in the model, the game is basically over at the moment. But, we may have a workaround by combining traditional databases with these new AI systems to mitigate the issues of not being able to delete data from a model.

Sirkka: The right to be forgotten assumes that people are interested in managing their data. What evidence do we have that individuals are motivated to spend time managing their information? It is also very challenging for individuals to articulate how they want organizations or their models to unlearn their data, not only at present but also from both the past and the future. In fact, temporality in individual data management is an open research field in IS. But as I indicated earlier, one cannot address individual issues, including unlearning and the right to be forgotten, without considering collective or group implications. If I delete my health data, what are the missed opportunities for a collective or group that I am part of?

Above, we talked about research and AI literacy. Alexander mentioned that we need new forms of literacy. But what are we losing as we invest in these new forms of AI literacy? What critical capabilities are we humans individually and collectively unlearning and forgetting as we increase our reliance on digital technologies such as AI?

Overall, the panelists underscored the importance of centering user well-being in AI research and development, particularly as AI technology reshapes the contours of individual identity, cognitive function and societal trust. The panelists emphasized that well-being must be understood in multidimensional terms, drawing from frameworks such as Seligman's PERMA model and Maslow's hierarchy, and measured dynamically, using methods like trace data and longitudinal studies. They pointed to risks such as overreliance on AI systems, cognitive offloading, exclusionary design and a growing digital divide, all of which challenge individuals' psychological, emotional and even moral agency. Notably, the discussion stressed the need to consider not only direct users of AI but also non-users and indirectly affected populations, whose opportunities and rights may be compromised by biased algorithms and opaque decision-making processes of AI systems.

The panelists also highlighted the critical role of researchers in bridging the gap between technical development and social responsibility. This includes informing policy and regulatory efforts, ensuring ethical design by embedding human values early in development and fostering AI literacy to strengthen users' critical engagement with algorithmic systems. It was noted that historical technological transitions that neglected individual well-being led to profound societal upheavals – a cautionary tale for today's accelerated pace of AI innovation. Moving forward, IS scholars are uniquely positioned to study and shape the interplay between users, developers and AI itself as dynamic agents within a broader socio-technical ecosystem. By doing so, they can help ensure that AI not only enhances efficiency but also upholds the dignity, purpose and well-being of its users and non-users who are directly or indirectly affected.