Use of chatbots for customer service in MSMEs Open Access

Today, conversational interfaces serve as a communication bridge between companies and customers. The way of communicating has changed, and human-computer interaction systems, commonly known as chatbots or conversational agents, support this process. Chatbots are a common emerging technology galvanized by artificial intelligence (AI) and machine learning with unparalleled business potential [1]. Chatbots represent one of the most relevant trends within communication settings and managing relationships with consumers. Chatbots are computer programs that allow interaction with users through a conversational interface and the use of natural language [2].

Nowadays, banks, businesses, the tourism industry, the education sector, the medical sector and governments, among others, have adopted chatbots as business development and competitiveness strategies [3–6]. Numerous micro, small and medium enterprises (MSMEs), such as hotels, online retailers, educational institutes, fitness centers, bakeries, stationery stores and beauty salons, have started using chatbot technology to among other things: improve customer communication, optimize staff time, help improve customer satisfaction, assist customers in making purchasing decisions, provide information about services or products and answer frequently asked questions, thereby enhancing their service innovation practices in order to gain better competitive advantage. Little by little, MSMEs in Ecuador are incorporating chatbots to improve customer service.

The pandemic highlighted the need for fast, remote and online customer service, which is one of the business benefits of implementing a chatbot. During the pandemic, many businesses prioritized digitalization to implement chatbots to automate routine tasks and improve customer service. The chatbot not only responds to customer questions and inquiries 24 hours a day, seven days a week, provides information instantly and interacts with customers, but it also helps collect customer behavior data to analyze trends based on customers' interests. Using chatbots increases business effectiveness and enhances operations, management and marketing [7, 8].

IT related entities have enabled investments in developing frameworks for the development of chatbots. Such tools can be made available to users via a variety of communication channels, including messaging chat services and many others; and traditional channels such as web chat, short message service (SMS) and voice calls [9]. Most MSME businesses use social media platforms to sell their products and provide customer service via chatbots. Social media and chatbots are very influential in helping MSME businesses communicate or strike a deal with customers. Chatbots could utilize social media as immediate response hubs for customer inquiries [10]. On the other hand, predicting appropriate responses to the user input is possible due to models constructed from large conversational datasets [11]. Nowadays, opening large-scale generative models such as Generative Pre-trained Transformer 3 (GPT-3) by OpenAI or bidirectional encoder representations from transformers (BERT) by Google has the potential to accelerate chatbot development further [12].

The System Usability Scale (SUS) is the most widely used standardized questionnaire for the assessment of perceived usability. In its standard (most often used) form, the SUS has 10 five-point items with alternating positive and negative tone [13]. The study of usability allows the evaluation of a chatbot to determine if it can be used by the user, or if the software required some changes to improve.

This research aims to contribute to analyze the usability of chatbots and determine how a chatbot is also beneficial for MSMEs, particularly for customer service. The usability testing allows for validation in a real environment of the functioning of the chatbot to verify the correct functioning of the dialog flows and the fulfillment of the proposed objectives. SUS is used to assess the perceived usability of chatbots implemented, which have been developed using the IBM Watson™ Assistant and DialogFlow frameworks. The architecture used for the development of chatbots is also presented.

Nowadays, online services in MSMEs are an essential means of interacting with customers for customer service, promotion or to provide information about the company's activities. Chat-based applications can provide different responses to user requests or questions [14, 15]. Singh and Beniwal [16] reviewed the different research on the conceptual architecture of chatbots. Some frameworks do not require programming knowledge, and the chatbot can be built from scratch through graphical tools.

Use and popularity of chatbots have steadily increased in recent years [17–19]. Chatbots are implemented in various areas, mainly for customer service [20]. Chatbots are accepted in the enterprise context for ease of use and productivity benefits [21]. Chatbots for customer service are attractive to businesses because they are always accessible to customers. For example, Domino's uses chatbots in some of its markets so that customers can interact with the restaurant and order pizza [22]. Similarly, in [23] the authors present a customer service chatbot that utilizes large-scale e-commerce data to respond to frequently asked questions, improving the customer's online shopping experience. The change in the business model in Nigeria, the use and implementation of the chatbot as a tool for digital transformation, and an improvement in the user experience and the social need for financial inclusion [24]. Bavaresco et al. [25] presented a systematic review of the literature in the field of business, and this research shows computational challenges, domains and methods focused on human-like conversational agents; for example, opportunities in areas such as commerce, finance, questions and answers in restaurants, medical domains and many others will be great challenges in the future. There is a significant impact on the acceptance of intelligent systems that support decision-making. Recently, Social, Mobile, Analytics and Cloud (SMAC) has caught the attention of various domains. For example, the banking area uses chatbot services in real-time; this allows us to get to know the client better, thus offering strategies to improve the relationship [26, 27].

The added value of chatbots is that the presence of a human is not necessary [28]. Chung et al. [29] identify in chatbots an emerging digital marketing strategy that companies are increasingly implementing to adapt to the growing digitally oriented service world. In the marketing industry, the central functions of chatbots have been identified as interaction, entertainment, fashion, personalization and problem-solving, which can also be interpreted as customer-related chatbot functions.

Chatbots may be used to interact with customers [30–32]. For example, during the prepurchase, chatbots that use machine learning algorithms and predictive modeling can instantly match a consumer's inquiry to available products that satisfy their needs [33]. During the purchase phase, chatbots might guide customers to shopping sites or present promotional offers [34]. In the postpurchase phase, the consumer can continue interacting with the chatbot to track the delivery process and establish relationships for after-sales care. Therefore, chatbots can play a significant role in client retention, particularly for vulnerable businesses such as MSMEs.

The dimension of chatbot quality consists of the five dimensions of response time, usability, reliability, availability and adaptability to measure the chatbot's technical success. Response time refers to the length of time before a message is responded to. Usability is about the ease of use of the chatbot. The chatbot needs to be reliable to have the possibility to use it anytime and anywhere. Trivedi [35] mentions that consumers must feel that using chatbots is easy; otherwise, the customer experience may be negatively affected. Adaptability indicatives of the ability to keep up with the changing developments.

A study has reported the six features of chatbot for effective adoption in diverse SMEs: (1) fast and accessible, (2) convenience, (3) natural dialog, (4) able to make a recommendation, (5) large data handling and (6) promote customer actions [1]. Adoption of implementing chatbots in SMEs has provided cost and time-saving opportunities and improved customer experiences [36]. However, to enhance the customer experience, the chatbot must have features like adaptability, availability, response time and usability quality [37]. One of the reasons for the introduction of chatbots is to provide a 24/7 enhanced live channel customer experience [38] noting that the customer experience in businesses utilizing chatbots is influenced by these bots' overall system design, customers' ability to use technology and customer trust in the brand and system.

In addition, there is a growing body of knowledge regarding methods and metrics for assessing chatbot user experience. User-centered evaluation has been central to research in several disciplines at the origins of chatbot research, such as using user satisfaction measures to evaluate dialog systems [39]. In the evaluation of chatbot research, instruments for users' self-reports of user experience, user observation and interviews [40], and analyses of chatbot interaction [41] are utilized.

An overview interdisciplinary suggests challenges a future study direction research on chatbots [12]. Some of the subjects covered in this significant research are shown in Table 1.

This section describes the methodology used to conduct the study, focusing primarily on participant information, chatbot development platforms, software development methodology and usability analysis tools. The research was primarily carried out to meet two important objectives: (1) develop new communication channels for customer service in MSMEs, such as chatbots (2) assessing the usability of implemented chatbots.

To analyze results about the use of chatbots, the chatbots Hatuncito, KinEc and botMGR are considered. The chatbots were developed and implemented in small companies. The Hatuncito chatbot was developed for a gastronomic training center. The chatbot is integrated into the institute's website, interacts with users and provides adequate and immediate answers about courses offered, learning contents, requirements, addresses, payment methods and course duration, among other topics. The KinEc chatbot responded to a request from the KinEc specialized physiotherapy center. The chatbot provides information to customers via WhatsApp. It allows users to consult topics on physical rehabilitation, physiotherapy, types of therapies, nutrition, opening hours, location and costs, among other topics. The botMGR chatbot was developed as a support system for the company MGR's online shopping and customer service area. The chatbot is integrated into the company's website, and answers frequently asked questions about products, engineering services and contacts.

To validate the correct functioning of the chatbots, SUS was applied. In the Hatuncito chatbot's case, 40 respondents participated, including managers, employees and customers. In the case of the KinEc chatbot and botMGR, the selected sample was 30 volunteer participants. In addition, three research professors from the Department of Computer Science of the Universidad Técnica Particular de Loja were actively participating.

Several chatbot development platforms are robust enough to be implemented in most companies, such as Dialogflow, Microsoft Bot Framework, IBM Watson™ Assistant, Amazon Lex, RASA, SAP, and many more [9, 42]. The IBM WatsonTM Assistant and DialogFlow frameworks were utilized to develop the chatbots in this study. Dialogflow is a natural language understanding platform that makes it easy to develop conversational agents and integrate them with mobile apps, web apps, devices, bots, interactive voice response systems and more. IBM Watson™ Assistant allows the construction of cross-platform chatbots using natural language through dialog knowledge, training data and logic that allows the assistant to interpret the user's request [43]. In general, the existing platforms with and without programming cover a broad spectrum of possibilities to facilitate the creation of chatbots in different scenarios.

The chatbots were implemented using agile development methodologies. In agile methodologies, software development is incremental, cooperative, simple and adapted. In addition, agile methodologies allow the client to be part of the development team and facilitate communication to achieve efficiency and adaptability to possible changes produced during the project's development [44].

Regarding agile software development methodologies, extreme programming (XP) was used for the Hatuncito and KinEC chatbots, which uses an object-oriented approach as a development paradigm, which encompasses a set of rules and practices that occur in the context of four structural activities: planning, design, coding and testing [45]. On the other hand, for the development of the botMGR chatbot, Scrum was used, which adapts to any project, since it has an iterative and incremental project management method, which has the following phases: product backlog, sprint planning, meeting, sprint backlog, daily standup meeting, sprint review and sprint retrospective [46].

Chatbot user experience and design concerns how users perceive and respond to chatbots. It relates to the users’ perception and response to chatbots. Conversational interfaces, interaction mechanisms and content should be designed to address user experience and behavior. User-centered evaluations of chatbots are necessary to collect insights and measure chatbots’ impact conducted through established methods [12].

A tool for usability analysis is the SUS, which helps determine a system's accuracy, functionality, effectiveness and satisfaction. The SUS (shown in Table 2) is a questionnaire with ten items, each with five scale steps. The odd-numbered items have a positive tone; the tone of the even-numbered items is negative. The SUS scoring method requires participants to provide a response to all ten items. Responses range from strongly agree to strongly disagree. If for some reason participants can’t respond to an item, they should select the center point of the scale. To get the overall SUS score, multiply the sum of the item score contributions by 2.5. Thus, overall SUS scores range from 0 to 100 [13].

The fundamental goal of usability testing is to help developers produce more useable products. Usability testing remains a central way of determining whether users accomplish their goals. Usability testing helps determine user satisfaction with the chatbot, including complexity, ease of use, dialog flow functioning, consistency, reliability and integration [47].

Typical chatbot solution architecture comprises multiple layers: presentation, business, service and data [48, 49]. The proposed architecture comprises three layers: presentation, business and data, which communicate through the hypertext transfer protocol (HTTP) protocol. Figure 1 shows the proposed architecture for developing chatbots. The main modules are the user interface; chatbot engine; node.js and webhook which act as a link between the chatbot engine and the database.

The user interface represents the front-end of the chatbot. Users can interact with chatbots from the computer, smartphone or tablet, through text or speech or links on the platform they are deployed on.

The chatbot engine is considered one of a chatbot's most important elements. Different frameworks can be used to build chatbots. The chatbot engine enables the agent to work with user input. The intent matching and entities recognition modules form the chatbot engine. In addition, the chatbot engine represents the chatbot system's backbone and should be independent of the used approach (natural language processing (NLP) or natural language understanding (NLU)).

Several approaches enable calling services from chatbots to handle inputs and outputs related to system integration, such as text to speech (TTS) or speech to text (STT), as appropriate.

A webhook can be created in any server-side language, such as Python or Node.js. Node.js is highly scalable because it can handle many simultaneous connections with high output. POST and GET requests from HTTP are routed by Node.js to get a specific response. A custom node.js was implemented, acting as a Webhook receiving information about the user's query and sending back response data. A webhook is a user-defined HTTP callback that can store and retrieve data from an event. Webhook is triggered when intent is matched, and a response is needed. The webhook interacts multiple times with the database until it retrieves all the required data to formulate the customized response and sends it back to the chatbot engine.

Conversational User Interface (CUI) is changing the way that we interact. CUI are platforms that mimic a conversation with a real human. Intelligent assistants, and chatbots, such as Alexa and Google Home, offer a new, natural, and intuitive human-machine interaction and open up a whole new world for humans [50].

The user interface is used to interact with the chatbot. It is composed of an area where the messages written by the user and the chatbot are displayed, a text field where the queries are entered, and a button to activate and deactivate the TTS converter. The chatbot starts the conversation with a welcome greeting and says what it can do so that visitors can ask the right questions and establish a dialog with successful results.

The dialog allows establishing a friendly relationship with the user, contains the guidelines to follow when establishing communication and provides a guide for defining the conversational flow of the chatbot. The dialog is characterized by brief and concise responses in simple language to make the interaction more human-like. Every conversation can have a global context and a specific context of the dialog. For example, talking about courses is the global context, and talking about a particular course is the conversation context.

A dialog manager has a set of plans, which can be divided into four groups: conversational-act determination and domain-task classification, intention identification, task processing and response generation [51].

The dialog manager considers the intent or motivation extracted from the user conversation to determine the appropriate action. It is trained to select an appropriate response by applying NLP and NLU. Feine et al. [52] consider the advantages of training and evaluating conversational agents based on artificial datasets. The advantages are as follows: (a) the dialogs resemble spontaneous human language; (b) the dialogs are easy to follow and contain less garbling and repetition; (c) there is a diversity of dialogs, topics, environments, actors and relationships.

Most chatbot frameworks specify intents with training phrases. To ensure that user input triggers the correct intents, training phrases must be included. Table 3 shows training phrases and response when an intent is created, for example, for ‘payment methods’ intent it is important to include training phrases such as “How do I pay?” to find out how to pay. Each intent contains training phrases, contexts and responses.

To create intents, a good practice when you provide training phrases is to have at least 10 to 15 different examples, depending on your intent’s complexity. Training phrases will be labeled with an intent name by a conversational designer. Once a chatbot is in production, the machine learning model can match a user utterance with the defined intent based on the training phrases the model was trained on.

When a user writes or says something in a chatbot, the dialog manager matches the expression to your best intent based on the training phrases and the built-in NLP. It’s possible to specify multiple different responses. The dialog manager will alternate through the response selection, so your answer could be unique each time.

Nuruzzaman et al. [53] highlighted that response generated-based chatbots are classified based on what action they perform in response generation. The response models take input and output in natural language text. The dialog manager is responsible for combining response models. To generate a response, the dialog manager follows three steps. First, it uses all response models to generate a set of responses. Second, it returns a response based on priority. Third, if there is no priority response, the model selection policy selects the response.

A study was conducted to analyze the usability of three chatbots implemented by MSMEs. Usability is one of the key factors for successful technology adoption. For the analysis of results, the SUS is used, a tool that allows knowing the system's usability from the point of view of a user [13]. SUS is easy to administer and has good reliability and validity measures; it also has substantial benchmarks to aid in interpreting scores [54]. Table 4 shows the percentiles range, grade, adjectives and net promoter score (NPS) categories to describe the raw SUS scores of the chatbots under study.

Grades range from A, which indicates superior performance, to F (for failing performance) with C indicating “average”. Adjectives allow you to use words instead of numbers to describe a customer experience. The scale contains adjectives including “Excellent”, “Good”, “Ok” and “Poor” – words users loosely associate with the usability of a product. Another variation in using words to describe the SUS is to think in terms of acceptability, for example, what is “acceptable”, “marginal” or “not acceptable”; acceptable corresponds to roughly above 70 and unacceptable to below 50. The range between 50 and 70 is marginally acceptable. The NPS is a management score to calculate customer loyalty. It is calculated based on responses to a single question: How likely is it that you would recommend our company/product/service to a friend or colleague? [13].

When applying the SUS questionnaire to know the usability of chatbots in the implemented case studies, scores of 93 were obtained for Hatuncito, 72 for KinEc and 85 for BotMGR. Most projects should aim for a SUS score of at least 80. Figure 2 presents the scores obtained, and the perception of chatbots for MSMEs is observed. The users' perception about the level of satisfaction with the use of chatbots shows that they would recommend their use in the case of the Hatuncito and BotMGR services considered “Excellent”. In addition, the KinEc chatbot is rated as “Good” in this case study for the specialized physiotherapy center, and the result reflects that customer experience is influenced by customers' ability to use technology because many of the people who interacted and evaluated the usability of the chatbot were older adults whom they have difficulty adopting new technologies and may perceive technology differently than younger adults do. In general, all chatbots were found to be acceptable; therefore, chatbots are more useful to MSMEs with consumer-facing businesses, as they tend to receive more customer support requests that are repetitive. Finally, from the usability analysis, it is clear that MSME managers should focus on making chatbots simple and easy to use. In particular, they should ensure that the chatbot can handle various tasks, including customer service, marketing and sales. They should also ensure that the chatbots provide clear and concise answers to users' questions.

Despite the encouraging findings of this study, there are limitations. Although chatbots are adequate for customer service, they may hinder the development of more natural, dynamic conversational agents. Further research should employ more sophisticated NLP techniques to develop chatbots with the capacity to engage in more natural, casual conversations. Lastly, this study did not consider the technological and experience biases that can influence perceptions and expectations. For example, participants may be biased if they have more (or fewer) experiences with chatbots or interact with them frequently. Future research should incorporate questions that consider potential technology bias to address this issue. Finally, advances in AI technology and large language models should be leveraged for the development and use of chatbots, including general-purpose discussion, information search, question answering and the like.

In conclusion, this study shows that MSMEs can use chatbots to reduce personnel costs, improve their internal processes and identify needs to improve customer relationships. The challenge is to adopt the integration of innovative and creative technologies such as chatbots without losing the essence of the company. Findings reflect that many MSMEs in Ecuador are applying chatbots to improve customer service. However, there are differences regarding the platforms, customization and responsiveness of the chatbot.

SUS questionnaire measures chatbot satisfaction. According to the results, implementing a user-oriented system allows parallel and interactive communication, providing immediate responses to user needs. The study's chatbots are widely accepted, which encourages MSMEs to integrate them to provide quick responses and reach a broader customer base. Chatbots provide instant information and services to users. Usability testing shows that chatbots can automate tasks, provide information and help with customer service. They should also implement chatbots carefully to avoid frustrating users.

Using a dialog manager in chatbot dialog design allows for constant learning system feedback and system optimization. If a question isn't in the knowledge base, the chatbot can send it to a human personal assistant. The related works show that chatbots are used in several fields. To benefit from this technology's developments, we must be clear about the project's goal, domain and direction.

Usability testing verifies the chatbot's question and response understanding. SUS is the most popular measure of perceived usability and will likely remain so. Chatbot frameworks and platform advancements are also key to chatbot interest. Large-scale generative models such as GPT-3 enable the creation of improved chatbots that can have more humanlike conversations.

The authors thank the Universidad Técnica Particular de Loja for the endorsement of the development of this research. Also, to the professors and students who contributed to implementing and validating the chatbot.