Elsevier

Information & Management 信息与管理

Volume 61, Issue 6, September 2024, 104012
第 61 卷,第 6 期,2024 年 9 月,104012
Information & Management

Exploring continued usage of an AI teaching assistant among university students: A temporal distance perspective
探索大学学生中 AI 教学助手持续使用的视角:时间距离视角

https://doi.org/10.1016/j.im.2024.104012Get rights and content 获取权利和内容

Abstract 摘要

Although technological developments have made AI chatbot teaching assistants a lived reality, limited insights exist indicating how students perceive and use these new technologies. Recognizing that technology usage research adopts a static view that overlooks the role of temporal distance, we draw on temporal construal theory to examine user intentions and actual usage behavior over time by re-approaching the same set of participants after developing and deploying an AI chatbot in the educational setting. Our results highlight the significance of temporal distance in shaping user perceptions of the technology, with the need for interaction level to play a moderating role.
尽管技术发展使人工智能聊天机器人教学助手成为现实,但关于学生如何感知和使用这些新技术,现有的见解有限。认识到技术使用研究采用了一种静态的观点,忽略了时间距离的作用,我们借鉴时间构念理论,通过在教育和部署人工智能聊天机器人后重新接触同一组参与者,来考察用户意图和实际使用行为随时间的变化。我们的结果突出了时间距离在塑造用户对技术的感知中的重要性,并强调了交互层面需要发挥调节作用。

Keywords 关键词

Ai chatbot
Continued usage behavior
Temporal construal theory
Need for human interaction
Higher education setting

人工智能聊天机器人持续使用行为时间距离理论人际互动需求高等教育环境

1. Introduction 1. 简介

Artificial intelligence (AI) has ushered in an era of profound technological revolution [42]. One example is the surge in AI chatbot systems, especially in business settings such as customer support and human recruitment: Gartner predicted that AI would completely handle 15 % of customer services by 2021 [8]. However, holding user attention remains a real problem. According to Messiahdas et al. [40], in the customer service context, the initial drop-off is huge: “About 40 % of users never get past the first text and another 25 % drop off after the second message.” Therefore, it is critical to understand the continued usage behavior associated with AI chatbots.
人工智能(AI)引领了深刻的技术革命时代[42]。一个例子是 AI 聊天机器系统的激增,尤其是在客户支持和人力资源招聘等商业环境中:Gartner 预测,到 2021 年,AI 将完全处理 15%的客户服务[8]。然而,保持用户注意力仍然是一个真正的问题。根据 Messiahdas 等人[40]的研究,在客户服务背景下,初始流失率很高:“大约 40%的用户从未通过第一条信息,另外 25%的用户在第二条信息后流失。”因此,了解与 AI 聊天机器人相关的持续使用行为至关重要。
Conventional research in information systems (IS) primarily adopts a fixed lens to understand ongoing usage behavior. This approach typically relies on users’ initial perceptions during adoption to forecast their behavior after adoption. However, technology features and attributes may become more or less important for users as the temporal frame shifts from the adoption stage to the continued behavior stage. According to temporal construal theory, the perception of temporal distance significantly shapes individuals' interpretations of events occurring at different points in time [67]. This interpretive process is achieved through either primary and goal oriented (high construal) or secondary and goal irrelevant (low construal) cognitive processing. For instance, Trope and Liberman [66] asked participants to rate a radio they would buy tomorrow (near-future temporal distance) or in a year (distant future). In the distant-future condition, they found that participants were persuaded by the radio's performance on the primary feature of sound quality. In the near-future condition, participants were persuaded by the radio's performance on the secondary feature of the radio's clock. Similar temporal effects might exist in the IS field. Considering the temporal distance, users' initial perceptions during adoption might not effectively drive their continued usage behavior because certain technology features may hold less importance in evaluating long-term benefits. Consequently, this study aims to assess whether temporal construal theory can explain changes in user perceptions while they are using chatbot technology.
传统信息系统(IS)研究主要采用固定视角来理解持续的使用行为。这种方法通常依赖于用户在采用过程中的初始感知来预测他们采用后的行为。然而,随着时间框架从采用阶段转移到持续行为阶段,技术特性和属性对用户的重要性可能会增加或减少。根据时间构念理论,对时间距离的感知显著塑造了个体对不同时间点发生事件的解释[67]。这种解释过程是通过主要和目标导向(高构念)或次要和目标无关(低构念)的认知处理来实现的。例如,Trope 和 Liberman[66]要求参与者对明天(近未来时间距离)或一年后(遥远未来)要购买的收音机进行评分。在遥远未来条件下,他们发现参与者被收音机在声音质量这一主要特性上的表现所说服。在近未来条件下,参与者被收音机在收音机时钟这一次要特性上的表现所说服。 相似的时间效应可能在 IS 领域存在。考虑到时间距离,用户在采用时的初始感知可能无法有效驱动他们的持续使用行为,因为某些技术特性在评估长期利益时可能不那么重要。因此,本研究旨在评估时间构念理论是否可以解释用户在使用聊天机器人技术时的感知变化。
This study considers the educational context due to the potential advantages of using chatbots to provide students with support outside of class. The decreased number of teachers in the education system has made integrating advanced technology into educational settings essential [57]. Hence, universities are increasingly employing AI-powered educational chatbots to help students [10], making designers and practitioners eager to understand strategies that can engage users and encourage the continued usage of AI chatbots. Recognizing the scarcity of relevant studies, we apply temporal construal theory to understand students’ continued usage of AI teaching assistants to help guide the technology's implementation.
这项研究考虑到使用聊天机器人为学生提供课外支持可能带来的潜在优势。教育系统中教师数量的减少使得将先进技术整合到教育环境中变得至关重要[57]。因此,大学越来越多地采用 AI 驱动的教育聊天机器人来帮助学生[10],这使得设计师和实践者渴望了解能够吸引用户并鼓励持续使用 AI 聊天机器人的策略。认识到相关研究的稀缺性,我们应用时间构念理论来理解学生持续使用 AI 教学助手,以帮助指导技术的实施。
Regarding the factors that influence prospective value assessments of and subsequent behaviors toward AI teaching assistant chatbots, we emphasize the significance of technological attributes and qualities. Although previous studies have highlighted perceived intellectual quality [[7], [31]] and perceived empathy [8] as AI chatbot attributes that influence user responses, a considerable number of these studies overlook temporal distance when considering users' evaluations of technology attributes. Adopting the criteria of primary/secondary and goal-relevant/irrelevant in classifying the construal level [67], we hypothesize that perceived intelligence aligns with high-level construal and positively influences assessments of the technology's distant-future usage value. In contrast, perceived empathy can be categorized as a secondary attribute for AI teaching assistants. This classification is based on the nature of the AI teaching assistant as an educational tool, where intelligence quotient (IQ) is likely to align with primary utilitarian goals and emotional quotient (EQ) is more likely to represent a peripheral characteristic. According to temporal construal theory, when assessing future usage, users tend to prioritize primary and goal-related factors (e.g., perceived intelligence of the AI teaching assistant), with secondary and goal-irrelevant factors (e.g., perceived empathy of the AI teaching assistant) holding more sway in evaluations of the status quo.
关于影响对 AI 教学助手聊天机器人预期价值评估及其后续行为的因素,我们强调技术属性和品质的重要性。尽管先前的研究已经强调了感知智力质量[7],[31]和感知同理心[8]作为影响用户反应的 AI 聊天机器人属性,但相当多的这些研究在考虑用户对技术属性的评价时忽略了时间距离。采用将构建水平分类为初级/次级和目标相关/无关的准则[67],我们假设感知智力与高级构建相一致,并积极影响对技术未来远程使用价值的评估。相比之下,感知同理心可以归类为 AI 教学助手的次级属性。这种分类基于 AI 教学助手作为教育工具的本质,其中智商(IQ)可能与初级功利目标相一致,而情商(EQ)更可能代表一种外围特征。 根据时间构念理论,在评估未来使用时,用户倾向于优先考虑主要和目标相关因素(例如,对 AI 教学助手的感知智能),而次要和与目标无关的因素(例如,对 AI 教学助手的感知同理心)在评估现状时具有更大的影响力。
Beyond contextual factors, research on temporal orientation suggests that individual differences substantially shape user expectations [38]. Indeed, the classification of an attribute as high-level or low-level construal is not a fixed characteristic of an event but rather depends on the alignment between user goals (e.g., using AI for learning or for entertainment) and the attributes of the technology. Our study focuses on the significant individual characteristic known as the need for human interaction (NFI). Among high-NFI users, the central focus concerns interactions with the AI chatbot as a social companion. Consequently, their primary concerns and core interests are more closely associated with perceived empathy. In contrast, the primary consideration for low-NFI users likely pertains to the AI chatbot's utility as a learning tool. As such, goal-oriented factors hinge predominantly on the AI chatbot's IQ. Thus, in considering substituting technology for human agents in educational contexts, we propose NFI as a crucial moderating factor that influences user responses to technological attributes.
超越情境因素,关于时间取向的研究表明,个体差异在很大程度上塑造了用户的期望[38]。事实上,将属性归类为高级或低级理解并不是事件的一个固定特征,而是取决于用户目标(例如,使用 AI 进行学习或娱乐)与技术属性之间的对齐。我们的研究聚焦于被称为人类互动需求(NFI)的显著个体特征。在高 NFI 用户中,主要关注的是与 AI 聊天机器人作为社交伴侣的互动。因此,他们的主要关注点和核心兴趣与感知同理心更为紧密相关。相比之下,低 NFI 用户的主要考虑可能涉及 AI 聊天机器人作为学习工具的实用性。因此,以目标为导向的因素主要取决于 AI 聊天机器人的智商。因此,在考虑在教育环境中用技术替代人类代理时,我们提出 NFI 作为一个关键调节因素,它影响用户对技术属性的反应。
Furthermore, in the post-adoption phase, research has extensively investigated the influence of expectation confirmation and satisfaction on sustained user behavior [5]. However, recent studies advocate an alternative perspective to better understand ongoing behavior [71], particularly in the context of AI chatbot adoption. Accordingly, our study focuses on another essential aspect of the relationship between expectation confirmation and continued behavior, namely, commitment to the status quo, defined as a user's attachment to the existing relationship with the current technology [20]. We posit that when user expectations are validated during the adoption stage, they tend to develop an attachment to their existing interaction with the current technology, which influences their choice to continue using it. As highlighted by Nebel [46] and Polites and Karahanna [52], even fully rational users may choose to maintain the status quo technology usage, despite the existence of superior alternatives, due to their commitment to it. This commitment relationship significantly elucidates users' continued behavior, an aspect that has received limited attention. Furthermore, we consider the possibility that for high-NFI users, the influence of commitment to the status quo on their continued behavior is weaker because AI cannot fully replace genuine human interaction. Thus, our research not only considers a potentially pivotal determinant of post-adoption continued usage behavior but also introduces NFI as a moderating factor.
此外,在采用后阶段,研究广泛探讨了期望确认和满意度对持续用户行为的影响[5]。然而,最近的研究倡导一种替代观点,以更好地理解持续行为[71],尤其是在人工智能聊天机器人采用的情况下。因此,我们的研究专注于期望确认与持续行为之间关系的另一个基本方面,即对现状的承诺,定义为用户对当前技术与现有关系的依恋[20]。我们认为,在采用阶段验证用户期望时,他们往往会对其与当前技术的现有互动产生依恋,这影响了他们继续使用它的选择。正如 Nebel[46]和 Polites 及 Karahanna[52]所强调的,即使是非常理性的用户,也可能由于对现状的承诺而选择维持现状的技术使用,尽管存在更优越的替代品。这种承诺关系显著阐明了用户的持续行为,这是受到有限关注的方面。 此外,我们认为对于高 NFI 用户来说,对现状的承诺对其持续行为的影响较弱,因为 AI 无法完全取代真实的人类互动。因此,我们的研究不仅考虑了采用后持续使用行为的潜在关键决定因素,还引入了 NFI 作为调节因素。
This paper is organized as follows. After exploring the literature on influencing factors in the adoption and post-adoption stages, we detail the research model's development. Next, we present the research methodology and the data analysis results. Finally, we discuss the paper's findings and contributions.
本文结构如下。在探讨影响采用和采用后阶段文献的基础上,我们详细阐述了研究模型的发展。接下来,我们介绍研究方法和数据分析结果。最后,我们讨论本文的发现和贡献。

2. Literature review 2. 文献综述

2.1. Temporal perspectives at the adoption stage
2.1. 采纳阶段的时序视角

2.1.1. Temporal construal theory
2.1.1. 时间构念理论

According to temporal construal theory, people usually evaluate outcomes and make decisions based on the temporal distance to the target behavior, which refers to “how much time separates the perceiver's present time and the target event” [38]. The influencing factors vary as the temporal distance to the target behavior changes [68]. After examining the relevant literature, we have identified a fundamental premise of temporal construal theory: The perception of temporal distance significantly influences how individuals interpret events occurring at different points in time. This interpretation is achieved through a process known as construal.
根据时间构念理论,人们通常根据目标行为的时间距离来评估结果和做出决策,这指的是“感知者当前时间与目标事件之间相隔多少时间” [38]。随着目标行为的时间距离的变化,影响因素也会变化 [68]。在审查相关文献后,我们确定了时间构念理论的一个基本前提:时间距离的感知显著影响个体对不同时间发生的事件的解释。这种解释是通过一种称为构念的过程实现的。
According to Trope and Liberman [67], “low-level construals are relatively unstructured, secondary, goal-irrelevant, and contextualized representations” and “include subordinate and incidental features of events.” In contrast, high-level construals are “schematic, primary, goal-relevant, and decontextualized representations that extract the gist from the available information” [67] (see Table 1). When individuals adopt a high-level construal, their focus is on primary and goal-relevant attributes, whereas a low-level construal directs attention toward secondary and goal-irrelevant attributes [68]. For instance, Martin et al. [38] discovered that in a distant-future advertisement for a cell phone, participants were more influenced by primary attributes (e.g., 400 h of standby time and 9 h of talk time), indicating a high construal level. Conversely, in a near-future advertisement, participants were more persuaded by secondary attributes (e.g., calendar, internal antennae), reflecting a low construal level. Adopting a similar rationale and building on the foundational principles of temporal construal theory, this study posits that the impact of AI teaching assistant attributes will also fluctuate between short- and long-term frames.

Table 1. Distinguishing high-level and low-level construals [67].

High-level construalLow-level construal
AbstractConcrete
SimpleComplex
Structured, coherentUnstructured, incoherent
DecontextualizedContextualized
Primary, coreSecondary, surface
SuperordinateSubordinate
Goal relevantGoal irrelevant
To operationalize the temporal frame, most studies have employed two-dimensional manipulation, such as comparing day and year [53] or examining near-future benefits and distant-future benefits [[29], [38]]. To align with the concept of evaluating the benefits of using technology over both short- and long-term time frames, we have categorized the perceived value of technology use into “perceived status quo usage value” (representing short-term benefits) and “expected value of future usage” (representing long-term benefits). This categorization enables us to investigate how various factors influence user evaluations of the same event, taking into account different temporal distances. Perceived status quo usage value refers to a user's assessment of the value derived from the current usage of the technology with the incorporation of uncertainty [20] in the adoption stage. Perceived future usage value refers to a user's rational and psychological assessment of the value derived from the distant future use of technology with the incorporation of uncertainty in the post-adoption stage. Factors influencing the status quo and future usage values are covariant. For example, Bettiga and Lamberti [4] found that positive emotions engendered more short-term adoption-stage value evaluation, with the motivational impact decreasing over a longer time frame in the post-adoption period. In this study, user evaluations in the adoption stage refer to status quo usage value; evaluations in the post-adoption stage refer to perceived future usage value.

2.1.2. AI chatbot attributes

The literature generally examines AI chatbot attributes in terms of two dimensions: IQ and EQ (e.g., [[31], [59]]).
A chatbot's IQ capacity captures perceived intelligence, that is, the degree to which users believe that the AI chatbot can understand their problems, use the information it has to search for relevant answers, and answer questions efficiently and accurately [[7], [31]]. This approach has origins in the technology acceptance model. Davis [13] notes that the functionality (i.e., perceived usefulness) of a technology system significantly influences user acceptance of the technology. Furthermore, whether a technology finishes its tasks is critical to determining user cognitive and behavioral responses [[16], [72]]. For AI chatbots developed for e-learning, the core function is to provide students with accurate and relevant information. Whether the AI chatbot provides relevant information is positively associated with usage value evaluation. Applying the temporal construal theory, we use the primary/secondary and goal-relevant/irrelevant classification criteria to define perceived intelligence as a primary attribute of AI teaching assistants. Accordingly, we assert that a high level of perceived intelligence tends to elicit a high-level construal, with this having a more pronounced impact on the future value assessment of the AI teaching assistant.
A chatbot's EQ capacity captures perceived empathy, that is, the extent to which users perceive the chatbot to mimic the experience of human communication by identifying, understanding, and reacting to users’ thoughts, feelings, behaviors, and experiences [[31], [43], [51]]. The computers-are-social-actors paradigm suggests that humans react to machines with human characteristics just as they react to humans [[44], [45]] and unconsciously bring the behaviors, reactions, and interaction routines of human interaction into the process of human-to-computer interaction, even if they know that the computer does not have feelings or emotions [44]. The empathy “shown” by AI encourages users to experience positive emotions because of the increased level of interaction [36]. Using the primary/secondary and goal-relevant/irrelevant classification criteria, perceived empathy can be categorized as a secondary attribute for AI teaching assistants. This classification is based on the nature of the AI teaching assistant as an educational tool, in which EQ is more likely to represent a peripheral characteristic. Such peripheral characteristics represent low-level construal factors, which generally have more influence on short-term evaluations.
Some studies have devoted attention to enhancing the cognitive and emotional intelligence (IQ and EQ) of AI chatbots through design science methodologies, and others have used qualitative approaches to explore users' motivations [65]. However, a notable gap persists in the investigation of potential temporal effects on users' decision-making processes. Certain attributes may be effective in providing short-term benefits, commonly referred to as status quo benefits, yet their efficacy may not extend to delivering long-term advantages. This temporal dimension introduces a nuanced layer to the understanding of user interactions with AI chatbots, emphasizing the need for a comprehensive exploration of the temporal dynamics influencing user decisions to engage with an AI chatbot.

2.1.3. Need for human interaction

Temporal orientation studies highlight the significant influence of individual differences in shaping user expectations and requirements regarding products [38]. Our investigation emphasizes the impact of the need for human interaction (NFI) on their evaluations. NFI is defined as the desire to enjoy interpersonal contact with human beings [[56], [60]]. As discussed, users demonstrate different personality traits when interacting with an AI (e.g., a chatbot) or a human agent, leading to users with different levels of NFI to have different expectations and requirements of a chatbot's service quality (e.g., [60]). Higher NFI users have greater requirements for human interaction, and more human-like features of AI chatbots may satisfy their social desires. Once the machine is capable of greater emotional support, high-NFI users are likely to transfer their human interaction desire to the machine. Meanwhile, low-NFI users have less need for human-related functionality, making them more likely to see the AI chatbot as a “tool” than as a “social partner.”

2.2. Longitudinal perspective of continued usage behavior

2.2.1. Continued intention and behavior

According to Fishbein [18], intention represents the motivation of users, which positively impacts subsequent behavior. Most studies have used continued intention in the adoption stage to represent the continued behavior of users in the post-adoption stage [47]. However, although intention can provide some insight into actual behavior, the real behavior in the post adoption stage is also influenced by other factors like the user's experience during the post-adoption stage. Therefore, longitudinal studies are needed to understand actual usage behavior more deeply.

2.2.2. Antecedents of continued behavior

According to Bhattacherjee [5], expectation confirmation strongly influences users’ subsequent behavioral responses. Expectation confirmation measures the extent to which users perceive a system's actual performance as aligning with their anticipated performance [5] and serves as a critical determinant of users’ cognitive attitudes toward the technology, such as their satisfaction and subsequent usage behavior. Existing studies have focused primarily on the effects of expectation confirmation on satisfaction and trust [[28], [61]]. To contribute fresh insights into the post-adoption stage, there is a call for a novel theoretical perspective to comprehend users’ ongoing usage behavior, as emphasized by Venkatesh et al. [71]. In response, we have explored an additional critical mechanism in the relationship between expectation confirmation and continued behavior, namely, commitment to the status quo. This investigation demonstrates that once their expectations are confirmed, users are likely to establish a strong attachment to a technology.
As noted by Nebel [46] and Polites and Karahanna [52], fully rational users sometimes choose to maintain the status quo even when superior alternatives exist. For such users, emotional factors, such as personal attachment, may exert a strong influence on their decision-making process. Bendapudi and Berry [3] and Sweeney and Soutar [64] have supported this notion by arguing that consumers often exhibit commitment to their current product because of the economic, social, or psychological costs of changing. In the context of the body of literature on technology usage, commitment to the status quo has been acknowledged as a crucial determinant of continued intention and behavior [20].
This effect may also be contingent on user NFI. For high-NFI users, given their strong desire for human interaction, the effect of commitment to the status quo in technology may be diminished because AI cannot fully replace genuine human interaction. This effect enables this study to consider a crucial-yet-underexplored mechanism in the post-adoption stage, providing insights into continued usage behavior that considers the individual personality of the user.

3. Research model

Fig. 1 visualizes our research model, providing a comprehensive causal chain for predicting continued usage behavior. This model posits that the deliberate and rational decisions of users are shaped by their assessments of status quo usage value and future usage value as influenced by various attributes of AI. The model also suggests that user NFI crucially shapes the perception of technology when temporal aspects are considered.
Fig 1
  1. Download: Download high-res image (292KB)
  2. Download: Download full-size image

Fig. 1. Research model.

Furthermore, the model illustrates that, in the post-adoption stage, beyond continued intention, the confirmation of user expectations and the commitment to the current technology critically determine actual usage behavior. We also suggest that NFI guides the impact of commitment to the status quo on continued behavior in the post-adoption stage.

3.1. The influence of AI chatbot attributes

The literature has extensively explored the correlation between functionality and perceived value [[16], [72]]. When an AI chatbot excels at providing accurate and efficient answers to user queries, user evaluations suggest high levels of commitment to the status quo and future usage intention. Conversely, if the AI chatbot struggles to perform its tasks effectively, the technology's utility is compromised, diminishing both perceived status quo usage value and perceived future usage value. Therefore, we posit that the AI chatbot's performance is positively linked to perceived status quo usage value and perceived future usage value.
Extensive research in various contexts has explored the favorable impact of perceived empathy in AI chatbots on user evaluations and behavior [[14], [35], [36], [58]]. For instance, Diederich et al. [14] and Liu and Sundar [35] have provided evidence that customers find empathic agents more likable and trustworthy because these bots exhibit human-like characteristics. In the educational setting, we propose that an AI chatbot's perceived EQ significantly influences user experiences and, consequently, their current and future usage behaviors. When students interact with an empathetic AI chatbot, they tend to appraise the AI chatbot's EQ more positively, enhancing its perceived value. Thus, we propose that the perceived empathy of an AI chatbot significantly influences the value derived from usage behavior.
Drawing on insights from temporal construal theory [68], we contend that perceived future usage value is more likely to be influenced by the perceived intelligence than the perceived empathy of an AI chatbot because events in the more distant future are typically associated with primary attributes [[38], [66]], signaling high-level construal. Given our interest in AI chatbots designed for e-learning purposes, their capacity to provide knowledge and answer questions aligns more closely with the user's primary objective of learning. Therefore, when the AI chatbot excels at answering questions, users are likely to positively evaluate its future usage value, perceiving it as a valuable tool for problem-solving and delivering personalized knowledge to students, that is, producing high-level construal.
Conversely, we argue that perceived status quo usage value is more likely to be influenced by the AI chatbot's perceived EQ. In the context of using an AI chatbot for educational purposes, its EQ is more likely to be a secondary attribute, producing low-level construal. Hence, EQ plays a more significant role in the context of a short temporal distance because it is a peripheral attribute. In support of this idea, Wilamowitz-Moellendorff et al. [73] found that perceived emotional benefits of a product lose relevance over time. In contrast, the impact of usefulness tends to increase over time [[30], [73]]. Applying the same rationale, we propose that an AI chatbot's EQ is more critical for short-term usage value in the adoption stage. However, its importance is likely to diminish over time. For long-term evaluations in the post-adoption stage, users are more likely to concentrate on the AI chatbot's functionality. This discussion leads to the following hypotheses:
  • H1a: In the adoption stage, the perceived intelligence of the AI chatbot will positively impact perceived status quo usage value.
  • H1b: In the adoption stage, the perceived intelligence of the AI chatbot will positively impact perceived future usage value.
  • H2a: In the adoption stage, the perceived empathy of the AI chatbot is a stronger predictor of the perceived status quo usage value than perceived intelligence.
  • H2b: In the adoption stage, the perceived empathy of the AI chatbot is a weaker predictor of the perceived future usage value than perceived intelligence.

3.2. The moderating effect of NFI

We propose that the user's NFI amplifies the impact of perceived EQ and diminishes the impact of IQ on perceptions of the value of AI chatbot usage. Higher NFI users have a stronger desire for human interaction and may find that the AI chatbot's human-like features satisfy their social needs. Conversely, lower NFI users have less demand for human-related functionalities and place greater emphasis on perceived intelligence.
This study focuses on the moderating effect of NFI on the relationship between AI attributes and the expected value of future usage compared to status quo usage. This focus stems from the expectation that various user groups may attribute greater significance to their primary and core considerations, increasing discernible disparities in their expectation of future value. Temporal construal theory posits that, in the short time frame, individuals tend to adopt a low-level construal, wherein their attention is directed toward secondary and goal-irrelevant attributes [68]. For high NFI users, their primary goal is linked to the perceived empathy of the AI chatbot, whereas for low NFI users, their primary goal is associated with the perceived intelligence of the AI chatbot. Building on the logic that the influence of primary and core factors intensifies with the passage of time [66], we propose that the differences in impact between primary cues will increase in user groups (i.e., high and low NFI groups) when they consider longer temporal distances. In contrast, the differences between secondary cues for short-term value expectations may not be as pronounced. Consequently, we anticipate a particularly pronounced moderating effect of NFI on perceived future usage value compared to that on perceived status quo usage value. This leads to the following hypotheses:
  • H3a: The need for human interaction weakens the relationship between perceived intelligence and perceived future usage value.
  • H3b: The need for human interaction strengthens the relationship between perceived empathy and perceived future usage value.

3.3. The influence of perceived status quo usage value and perceived future usage value

We argue that users are rational decision-makers. Thus, they will spend time and effort to use the AI chatbot only when they perceive value from its use. If the value derived from current and future usage is limited, users are likely to abandon the usage behavior in the following period [[37], [75]]. Hence, both perceived current and future usage values are expected to impact user intention to continue using the technology. Furthermore, compared to perceived current usage value, perceived future value is likely to be a stronger predictor of continued usage intention because future behavioral intention is more likely to be associated with future perceptions in the post-adoption stage. As such, we propose that perceived future usage value more strongly predicts continued usage than perceived current usage value, leading to the following hypotheses:
  • H4a: In the adoption stage, perceived status quo usage value will positively impact the continued intention to use the AI chatbot.
  • H4b: In the adoption stage, perceived future usage value is a stronger predictor of continued intention to use the AI chatbot than perceived status quo usage value.
Next, we propose that the perceived status quo usage value significantly influences the perceived future usage value. That is, if users have a positive experience in their current use of the technology, they are more likely to anticipate a prolonged positive experience by the future use of the technology. This leads to the following hypothesis:
  • H5: Perceived status quo usage value in the adoption stage will positively impact perceived future usage value.

3.4. Post-adoption continued usage behavior

Drawing on Fishbein's model (1979), we recognize intention as a representation of user motivation that exerts a positive influence on subsequent behavior. To capture the dynamic interplay over time, Venkatesh and Davis [69] demonstrated the utility of using intentions measured at T1 to predict usage at T2, intentions at T2 to predict usage at T3, and so forth. Following a similar rationale, our study posits that continued usage intention during the adoption stage positively correlates with subsequent usage behavior during the post-adoption stage, leading to the following hypothesis:
  • H6: Continued usage intention in the adoption stage will positively impact the continued usage of the AI chatbot in the post-adoption stage.
Beyond continued intention, this study introduces another mechanism that influences continued usage behavior in the post-adoption stage, namely, the commitment to the status quo mechanism.
First, we contend that perceived future usage value during the adoption stage will negatively impact expectation confirmation in the post-adoption stage. According to expectation management theory [6], user expectations regarding the future performance of a new working system should align with the system's actual subsequent performance. Therefore, when users hold higher expectations for the post-adoption stage, there is a reduced likelihood of those expectations being positively confirmed in that stage. As noted by Buschmeyer et al. [6], it may even be advantageous for users to initially underestimate the product's performance and experience the positive surprise of, for example, discovering its usefulness. Aligning with expectancy confirmation theory, we propose that perceived future usage value at T1 will negatively affect user expectation confirmation at T2. Accordingly, we propose the following hypothesis:
  • H7: Perceived future usage value in the adoption stage will negatively impact expectation confirmation in the post-adoption stage.
Second, we posit that expectation confirmation will positively impact user commitment to the status quo in the post-adoption stage. Drawing on Goyal et al. [20], commitment primarily arises when users develop a sense of psychological ownership through repeated and interactive usage experiences, especially in the post-adoption stage. Therefore, we contend that commitment to the status quo is a gradual development that is highly likely to be fostered by the confirmation of expectations during usage. According to previous research, when a party consistently demonstrates reliability in line with user expectations, users tend to form stronger attachments to that party [32]. Consequently, users become more willing to prolong their relationship with the technology. Conversely, if the initial beliefs are not confirmed during post-adoption usage, user perceptions of the technology are adjusted, and they may consider discontinuing their relationship with it. Based on this rationale, we propose the following hypothesis:
  • H8: In the post-adoption stage, expectation confirmation will positively impact commitment to the status quo.
Third, the commitment to the status quo that users develop during the post-adoption stage crucially contributes to their continued use of the AI chatbot. The literature highlights a significant, positive impact of commitment to the status quo on subsequent continued usage behavior [20]. Users may hesitate to abandon the incumbent technology due to their fear of losing the enjoyable experience, familiarity, and emotional attachment associated with the technology. Therefore, on establishing a commitment to the status quo, users become more inclined to continue using the technology. Accordingly, we propose the following hypothesis:
  • H9: In the post-adoption stage, commitment to the status quo will positively impact continued usage of the AI chatbot.
Finally, we propose that NFI weakens the influence of commitment to the status quo on continued usage behavior. The effect of commitment to the status quo on continued use will be less pronounced for high-NFI users because AI cannot fully replace humans, especially in interaction-related tasks. This rationale prompts the following hypothesis:
  • H10: In the post-adoption stage, the need for human interaction weakens the relationship between commitment to the status quo and continued usage behavior.

4. Research methodology

4.1. Data collection

To empirically assess our model, we conducted a longitudinal field study in a real-world setting. This involved deploying an AI chatbot named Teaching Assistant XiaoJun in the undergraduate course Introduction to Artificial Intelligence at a university in Northeast China.
Teaching Assistant XiaoJun was developed based on the Turing Robot platform, a prominent Chinese AI company focused on Natural Language Processing (NLP) and Chatbot service which already served more than 130,000 companies and developers globally. This Turing Robot platform is equipped with advanced NLP technologies and designed to mimic human thought processes and emotional recognition, enabling it to understand user questions effectively. Its responses are generated from an internal NLP model and augmented by a customized knowledge base. Our customized corpus included detailed course-specific content that allows Teaching Assistant XiaoJun to answer questions about the course textbook, extracurricular exercises, grading, final assessment requirements, course scheduling, and other relevant information (see Figure 2a).
Fig. 2a, Fig. 2b illustrate the process of interaction between students and the AI teaching assistant. It is worth noting that Teaching Assistant XiaoJun was seamlessly integrated with an official WeChat account. Students could easily engage with XiaoJun via WeChat, creating a chat experience akin to conversing with a WeChat friend. As depicted in Figure 2b, Teaching Assistant XiaoJun's extensive knowledge base allowed it to comprehend and respond to users’ emotional messages, mimicking human interaction. It is also important to mention that this version of the AI chatbot operated exclusively via text, with the profile photo of XiaoJun produced to represent a seemingly knowledgeable AI. The deliberate selection of a neutral name and image for the AI chatbot aims to mitigate any gender-related effects of the AI design on users' reactions.
Fig 2a
  1. Download: Download high-res image (707KB)
  2. Download: Download full-size image

Fig. 2a. AI teaching assistant XiaoJun user interface.

Fig 2b
  1. Download: Download high-res image (502KB)
  2. Download: Download full-size image

Fig. 2b. AI teaching assistant XiaoJun user interface (continued).

We introduced Teaching Assistant XiaoJun to students during the summer semester of 2022 (August 8 to September 4) in the undergraduate course Introduction to Artificial Intelligence at the study university. This introductory course focused not on technical skills, but on providing students with insights into the concepts and application scenarios of AI technology. Consequently, the level of difficulty for each section of the course was relatively consistent.
In terms of ethical considerations, we prioritized transparency in our approach. At the beginning of the course, all students received comprehensive information regarding the purpose of and the process for data collection. This upfront disclosure was a fundamental aspect of our ethical framework. Additionally, students were informed that their engagement with the AI teaching assistant was entirely voluntary. We placed great emphasis on ensuring that their participation was driven by personal choice and not influenced or coerced in any way.
We collected data related to the use of Teaching Assistant XiaoJun at two points during the semester to capture the influences of user intentions on actual usage. These two points were at the beginning of the semester during the first week's introductory classes (T1; the adoption stage) and at the end of the semester during the fourth week's classes (T2; the post-adoption stage). For each test, students were asked to input their student ID, enabling tracking of individual responses over time. Notably, students were not allowed to withdraw from the course after the first week.
Of the 179 students in the class, 137 had used the AI chatbot at T1, and 146 had used it at T2. To ensure that our analysis focused solely on genuine users, we introduced a qualifying question: “The Teaching Assistant XiaoJun for this course is _____: A. an AI chatbot; B. a senior; C. an assistant professor; D. a lecturer.” Additionally, we included a deceptive question (“Please select strongly disagree from the options”) in the middle of the questionnaire to filter out inattentive responders [[21], [26]]. Only the respondents who successfully passed both the qualifying item and the deceptive question were included in the subsequent analyses. Consequently, we identified 105 valid datasets at T1. However, four of these datasets did not pass the deceptive question at T2. Therefore, a total of 101 valid sets of responses were available. Of these, 38.6 % (N = 39) were male, 61.4 % (N = 62) were female, and all were in the age group 20–25 years old. Most were second-year undergraduate students. Detailed demographic information about the respondents is included in Table 2.

Table 2. Demographic information (N = 101).

ItemCategoryFrequencyPercentage
GenderMale3938.61 %
Female6261.39 %
YearYear 121.98 %
Year 29897.03 %
Year 300.00 %
Year 410.99 %
Academic performanceTop 10 %98.91 %
10 %−30 %3837.62 %
30 %−50 %2827.72 %
50 %−100 %2625.74 %

4.2. Measurement scales

Theoretical constructs were operationalized using validated items from previous research (see Table A1 in the Appendix). To assess perceived status quo usage value and perceived future usage value, we borrowed items from Goyal et al. [20]. For status quo and future usage values, we highlighted the words “at present” and “in the future” to emphasize the differences between the two constructs. Following Venkatesh et al. [70], we measured usage behavior by asking subjects, “How often do you use the AI chatbot?” Frequency was chosen to reflect the real usage behavior because of the query-based nature of the AI chatbot (e.g., [9]). All items were measured using a seven-point Likert scale ranging from “strongly disagree” (1) to “strongly agree” (7).
Notably, prior research has gauged expectation confirmation in three ways: objective, perceived, and inferred [5]. Objective confirmation involves an external evaluator assessing the expectation-performance gap. However, it has been criticized for its lack of accuracy because it does not consider variations in user expectations and post-usage perceptions. Perceived confirmation relies on consumers’ subjective post-use ratings of the expectation-performance match [49]. This approach has been criticized for its bias toward “post-decision dissonance reduction” [17], which sees subjects rationalize their choices to align with prior expectations. Inferred confirmation calculates the difference in ratings by consumers on selected attributes before and after their consumption experience [63]. This approach is preferred by scholars because it creates a composite confirmation index [[48], [54], [62]].
Similar to previous research, our study also uses the difference between customer perception and expectation ratings to understand post-adoption behavior. For the calculation process, previous studies determined expectation confirmation by subtracting pre-purchase expectations from post-purchase evaluations for each item [54]. Following a similar rationale, we computed expectation confirmation by subtracting adoption stage expectations from post-adoption stage evaluations for each item. Specifically, we subtract the perceived future usage value (T1, representing expectations) from the perceived status quo usage value (T2, reflecting performance). This subtraction yields a score indicating the extent of confirmation or disconfirmation of expectations. For example, in the case of EC1, we calculated the score by subtracting the perceived future usage value for PFUV1 (“In the future, Teaching Assistant XiaoJun will make me feel comfortable”) (T1) from the perceived status quo usage value for PSQV1 (“At present, Teaching Assistant XiaoJun makes me feel comfortable”) (T2) for each participant.
Furthermore, aligning with previous studies, we have employed control variables, including gender, prior experience, and academic performance [[1], [9]]. Additionally, trust and habits are considered as control variables based on the findings of Goyal et al. [20], which suggest that trust and habits significantly influence user commitment to the status quo and continued usage behavior.

5. Results

5.1. Testing the measurement model

To test the measurement model, we examined both the internal reliability and convergent and discriminant validity of each item in each construct. As Table 3 shows, factor loadings for each construct were all above 0.7, indicating acceptable convergent validity [[2], [25]]. Internal reliability was assessed by examining Cronbach's alpha, rho_A, and the composite reliability values of each construct. All relevant values exceeded the 0.7 threshold, implying satisfactory construct reliability [[15], [19]]. The average variance extracted (AVE) values for each construct were all above the minimum critical value of 0.5, indicating good convergent validity [11]. To check for potential multicollinearity between the influencing variables, especially for the constructs of perceived status quo usage value and perceived future usage value, we determined the variance inflation factor (VIF) value. According to Hair et al. [22], multicollinearity is present in the data if the VIF value exceeds 10. In this study, all the VIF values were below 4, connoting that multicollinearity is unlikely to be a concern.

Table 3. Reliability and convergent validity analysis.

ConstructItemsFactor loadingsCronbach's alphaComposite reliabilityrho_AAVEVIF
Perceived empathy (PEM)PEM10.7910.8540.9180.9080.7681.553
PEM20.910
PEM30.922
Perceived intelligence (PI)PI10.9100.8330.8620.8990.7501.673
PI20.895
PI30.787
Need for human interaction (NFI)NFI10.8080.8740.9340.9200.7941.221
NFI20.973
NFI30.885
Perceived status quo usage value (PSQV)PSQV10.9020.9080.9160.9420.8443.827
PSQV20.908
PSQV30.945
Perceived future usage value (PFUV)PFUV10.9300.9210.9280.9500.8653.641
PFUV20.894
PFUV30.965
Expectation confirmation (EC)EC10.9040.8860.8910.9290.8142.607
EC20.877
EC30.925
Commitment to status quo (CTSQ)CTSQ10.9100.8650.9180.8660.7891.929
CTSQ20.918
CTSQ30.835
Continued usage intention (CUI)CUI10.9620.9550.9710.9560.9182.713
CUI20.966
CUI30.946
Discriminant validity was assessed using the Heterotrait–Monotrait (HTMT) Ratio, which compares between-trait correlations and within-trait correlations. The HTMT approach—rather than classic approaches such as the Fornell-Larcker criterion—was adopted because HTMT outperforms alternatives in terms of assessing discriminant validity [24]. As Table 4 demonstrates, the HTMT value for each pair of constructs is below the 0.9 threshold for both the full sample and subgroups, suggesting acceptable discriminant validity.

Table 4. HTMT analysis results.

Empty CellPEMPINFIPSQVPFUVECCTSQ
Perceived empathy (PEM)
Perceived intelligence (PI)0.559
Need for human interaction (NFI)0.1530.271
Perceived status quo usage value (PSQV)0.5470.5540.083
Perceived future usage value (PFUV)0.4640.6100.1900.857
Expectancy confirmation (EC)0.1960.4140.1380.3100.478
Commitment to status quo (CTSQ)0.1430.1270.1750.3070.3520.386
Continued usage intention (CUI)0.2920.5580.4200.7060.7380.1850.378
Next, a common method bias test was conducted to reinforce remedial approaches and prevent biases resulting from self-reported measurements. Following Liang et al. [34], we conducted a statistical analysis by adding a common method factor that included all construct items as indicators. As Table 5 shows, most factor loadings are insignificant, with the substantive factor loadings remaining significant. Additionally, the average substantively explained indicator variance is 0.829, with the average method-based variance being 0.015, significantly lower than indicated by the substantive constructs. The ratio of substantive variance to method variance is approximately 55:1. The small magnitude and insignificant results for method variance enable the conclusion that common method bias is not a serious concern.

Table 5. Common method bias test results.

ConstructIndicatorSubstantive factor loading (R1)R1 squareMethod factor loading (R2)R2 square
Perceived empathy (PEM)PEM10.942***0.887−0.1800.032
PEM20.772***0.5960.182*0.033
PEM30.936***0.876−0.0220.000
Perceived intelligence (PI)PI10.937***0.878−0.0320.001
PI20.785***0.6160.1270.016
PI30.884***0.781−0.1080.012
Need for human interaction (NFI)NFI10.870***0.757−0.2020.041
NFI20.984***0.9680.0000.000
NFI30.806***0.650−0.0680.005
Perceived status quo usage value (PSQV)PSQV11.065***1.134−0.1860.035
PSQV20.904***0.8170.0080.000
PSQV30.801***0.6420.1620.026
Perceived future usage value (PFUV)PFUV11.200***1.440−0.2800.078
PFUV20.698***0.4870.2100.044
PFUV30.862***0.7430.1150.013
Expectancy confirmation (EC)EC10.923***0.8520.0670.004
EC20.870***0.757−0.0440.002
EC30.931***0.8670.0240.001
Commitment to status quo (CTSQ)CTSQ10.866***0.750−0.0920.008
CTSQ20.920***0.846−0.0350.001
CTSQ30.890***0.7920.0190.000
Continued usage intention (CUI)CUI10.994***0.988−0.0360.001
CUI20.992***0.984−0.0300.001
CUI30.888***0.7890.0660.004
Average0.9050.829−0.0140.015
Note(s): * refers to p < 0.05; *** refers to p < 0.001; the numbers in italics refer to the average substantive variance and the average method variance.

5.2. Testing the structural model

To evaluate the research model and test the hypotheses, we adopted a partial least squares structural equation modeling (PLS-SEM) approach using SmartPLS 4.0. We chose PLS-SEM rather than covariance-based SEM because the PLS approach is more suitable for small sample sizes [23]. Furthermore, PLS-SEM is appropriate for the exploratory stage of theory building and prediction [[23], [39]]. Given that this study not only tests existing theories but also expands the theoretical framework by adding new paths, PLS was deemed more suitable. Fig. 3 visualizes the results produced by the proposed model.
Fig 3
  1. Download: Download high-res image (264KB)
  2. Download: Download full-size image

Fig. 3. Structural model results.

To provide a detailed overview of the hypothesis testing results, we present Table 6. In the adoption stage, the AI chatbot's perceived intelligence significantly influenced the perceived status quo usage value (ß=0.293, p < 0.01) and perceived future usage value (ß=0.246, p < 0.01), supporting H1a and H1b. Furthermore, perceived empathy demonstrates a stronger impact on status quo value than perceived intelligence (ß=0.328, p < 0.001; Δb = 0.035), supporting H2a. The influence of perceived empathy on perceived future usage value is statistically insignificant, while perceived intelligence is found to be significant (ß=0.246, p < 0.01) and has a stronger effect on perceived future usage value (Δb = 0.232), supporting H2b.

Table 6. Hypothesis testing results.

Hypothesesß
(Standardized coefficient)
ΔbStandard errorsp-value95 % confidence intervalResults
LowerUpper
H1a: PI→PSQV0.293⁎⁎0.0350.0950.0020.1280.503Supported
H2a: PEM→PSQV0.328⁎⁎⁎0.0880.0000.1500.494Supported
H1b: PI→PFUV0.246⁎⁎0.2320.0800.0020.0620.382Supported
H2b: PEM→PFUV0.014ns0.0700.846−0.1330.156Supported
H4a: PSQV→CUI0.288*0.1800.1400.040−0.0130.534Supported
H4b: PFUV→CUI0.468⁎⁎0.1350.0010.2000.740Supported
H5: PSQV→PFUV0.758⁎⁎⁎NA0.0570.0000.6470.868Supported
H6: CUI→CUB0.646⁎⁎⁎NA0.0750.0000.4200.723Supported
H7: PFUV→EC−0.442⁎⁎⁎NA0.0970.000−0.620−0.239Supported
H8: EC→CTSQ0.337⁎⁎NA0.1000.0010.1380.536Supported
H9: CTSQ→CUB0.320⁎⁎NA0.1060.0030.1100.530Supported
Moderation analysis
NFI→PSQV−0.016nsNA0.1110.885−0.2400.221NA
NFI→PFUV−0.029nsNA0.0690.678−0.1790.080NA
NFI→CUB0.224nsNA0.1260.076−0.0250.473NA
H3a: PI×NFI→PFUV−0.295⁎⁎NA0.0860.001−0.416−0.083Supported
H3b: PEM×NFI→PFUV0.153*NA0.0720.0340.0010.274Supported
H10: CTSQ×NFI→CUB−0.170*NA0.0850.049−0.339−0.001Supported
Note(s): CUB refers to continued usage behavior; CUI refers to continued usage intention; PSQV refers to perceived status quo usage value; PFUV refers to perceived future usage value; PI refers to perceived intelligence; PEM refers to perceived empathy; NFI refers to need for human interaction; EC refers to expectation confirmation; CTSQ refers to commitment to status quo; NA refers to not applicable; * refers to p < 0.05; ** refers to p < 0.01; *** refers to p < 0.001; ns refers to not significant.
Regarding moderating effects, we observed NFI to significantly weaken the impact of perceived intelligence on perceived future usage value (ß=−0.295, p < 0.01), supporting H3a. Meanwhile, NFI demonstrated a positive moderating effect (ß=0.153, p < 0.05) on the relationship between EQ and perceived future usage value, supporting H3b.
In addition, the influence of perceived status quo usage value on continued usage intention is statistically significant (ß=0.288, p < 0.05), supporting H4a. Perceived future usage value more strongly predicted continued usage than perceived status quo usage value (ß=0.468, p < 0.01; Δb = 0.180), supporting H4b. Next, perceived status quo usage value strongly impacted perceived future usage value (ß=0.758, p < 0.001), supporting H5.
Regarding continued usage behavior in the post-adoption stage, continued usage intention formed in the previous period strongly influenced continued usage behavior (ß=0.646, p < 0.001), supporting H6. Meanwhile, perceived future usage value in the adoption stage demonstrated negative effects on expectation confirmation (ß=−0.442, p < 0.001), supporting H7. Next, expectation confirmation positively influenced commitment to the status quo (ß=0.337, p < 0.001), supporting H8 and thus encouraging continued usage behavior (ß=0.320, p < 0.01), supporting H9. Finally, we observed that NFI significantly weakens the impact of commitment to the status quo on continued usage behavior (ß=−0.170, p < 0.05), supporting H10.

5.3. Post hoc moderating effect test

NFI as a personality trait is stable and can be reflected in user preferences for machine or human interaction concerning the same task [55]. Given the interesting results produced by consideration of NFI, we adopted an alternative approach to examining its moderating effects. To measure user NFI at T2, we gave students two options for inquiring about the requirements for the final term paper: They could ask either Teaching Assistant XiaoJun or a human teaching assistant. The relevant information was input into the knowledge base beforehand, meaning both Teaching Assistant XiaoJun and the human teaching assistant would give the same response. After explaining the two options, we immediately collected data at T2, with NFI measured using the following item: “Please indicate whether you would choose Teaching Assistant XiaoJun or a human teaching assistant to inquire about the requirements of the final term paper.” Based on the definition of NFI provided by the literature (e.g., [[55], [60], [74] ]), we identified those who selected the human teaching assistant as high-NFI individuals and those who chose Teaching Assistant XiaoJun as low-NFI individuals. Upon identifying these subgroups, we conducted a sub-group analysis in accordance with the procedure introduced by Keil et al. [27]. The results of this analysis appear in Table 7.

Table 7. Moderating test: Subgroup analysis.

PathsSubgroupsß
(Standardized coefficient)
Standard errorsp-value95 % confidence intervaltspooledResults
LowerUpper
PI→PFUVLow NFI (N = 53)0.705⁎⁎⁎0.1260.0000.4560.95312.45H3a supported
High NFI (N = 48)0.228ns0.2570.376−0.2800.736
PEM→PFUVLow NFI (N = 53)−0.002ns0.1480.988−0.2940.289−14.70H3b supported
High NFI (N = 48)0.380⁎⁎0.1210.0020.1400.620
CTSQ→CUBLow NFI (N = 53)0.319⁎⁎0.0920.0010.1370.5014.23H10 supported
High NFI (N = 48)0.239*0.1130.0360.0150.463
Note(s): PI refers to perceived intelligence; PEM refers to perceived empathy; NFI refers to need for human interaction; PFUV refers to perceived future usage value; CTSQ refers to commitment to status quo; CUB refers to continued usage behavior; * refers to p < 0.05; ** refers to p < 0.01; *** refers to p < 0.001; ns refers to not significant.
As Table 7 indicates, perceived intelligence significantly influences perceived future usage value for the low-NFI group, with the significant effect disappearing for the high-NFI group. The results resemble those produced by the previous analysis, implying that the low-NFI group has greater expectations for the AI chatbot's functionality. Following the calculation procedure introduced by Keil et al. [27], we conducted a multi-group analysis, with the results providing empirical evidence for the substantial difference between the two subgroups. The effect of perceived intelligence on perceived future usage value was significantly greater for the low-NFI group than for the high-NFI group (ßlow-NFI=0.705, ßhigh-NFI=0.228, tspooled=12.45). The effect of perceived empathy on perceived future usage value was significantly greater for the high-NFI group than for the low-NFI group (ßlow-NFI=−0.002, ßhigh-NFI=0.380, tspooled=−14.70). The high-NFI group demonstrated a preference for the AI chatbot to have more human-like functionality. Therefore, our alternative approach also supports the significant moderating effect of user NFI on evaluations of the future usage value of the AI chatbot.
Furthermore, as Table 7 also shows, the path coefficients from commitment to the status quo to continued usage behavior differ significantly between the low-NFI user group and the high-NFI user group (ßlow-NFI=0.319, ßhigh-NFI=0.239, tspooled=4.23). This observation implies that high-NFI users may still exhibit hesitancy when it comes to maintaining their use of the AI teaching assistant, even if they are committed to their existing technology relationship. This alternative perspective further underscores the significant moderating influence of user NFI on the relationship between commitment to the status quo and continued usage behavior in the post-adoption stage.

6. Discussion and implications

6.1. Discussion

Incorporating temporal construal theory, we posit that users’ deliberate and rational decisions are shaped by assessments of status quo usage value and future usage value, which are influenced by various attributes of AI. NFI significantly moderates users’ perception of the technology, especially when temporal aspects are considered. Furthermore, in the post-adoption stage, apart from continued usage intention, the confirmation of user expectations and the establishment of commitment to the current technology forms an essential mechanism that determines actual usage behavior, with NFI again playing a pivotal moderating role.
First, through the lens of temporal construal theory, this study provides evidence that user evaluations of technological attributes depend on their temporal perspective. The primary and goal-relevant factor of perceived intelligence quality more strongly shapes perceptions of distant future usage value. Conversely, the secondary and goal-irrelevant factor of perceived empathy exerts a more significant influence on status quo usage value. These findings align with the observations of Trope and Liberman [66], showing that distant-future decisions prioritize primary features, with near-future decisions focusing on secondary features. Results show that the EQ of the AI chatbot does not significantly impact perceived future usage value, suggesting that hedonic factors become less influential in the post-adoption stage. In other words, for utilitarian AI, the EQ may offer only short-term benefits during the adoption phase, lacking a lasting impact. By integrating the concept of temporal distance into the analysis of AI chatbot usage behavior, this research not only substantiates the relevance of temporal construal theory in the context of AI chatbots but also provides practical insights for adjusting user engagement strategies over time.
Second, our findings emphasize the role of perceived status quo usage value and perceived future usage value as two distinct constructs in shaping user intentions to continue using technology. Notably, we identified that the strength of the determination of continued usage intention is greater when users consciously and rationally assess the value of future technology usage compared to their evaluation of current usage. These two constructs have traditionally been used interchangeably in the literature, whereas the approach might overlook the nuanced differences between the two constructs. This distinction holds practical implications for designing targeted interventions and strategies aimed at fostering enduring user engagement through the long-term benefits of attribute design.
Third, our investigation has revealed a significant moderation effect of user NFI on the impact of the AI chatbot's perceived intelligence and perceived empathy on perceived future usage value. Users with a low NFI may perceive an AI teaching assistant as a learning tool, emphasizing the AI's IQ in relation to their long-term objectives. Meanwhile, individuals with high NFI may regard the AI teaching assistant as a partner, more substantially intertwining their long-term aspirations with the AI's EQ. Our findings align with the literature asserting that user perceptions of an event are influenced by their individual characteristics [38]. In the context of AI chatbot usage, the characteristic of the need for human interaction serves as a pivotal individual difference that shapes the impact of technology attributes on users' responses.
Finally, by collecting actual usage behavior data, we have been able to recognize that users’ continued usage behavior in the post-adoption stage is not based solely on their previous intentions but is also influenced by their experiences after adoption. Specifically, the confirmation of user expectations significantly impacts their commitment to the status quo relationship with the chatbot, shaping their continued usage behavior. This observation aligns with previous findings [20], indicating that users' persistent behavior may stem from their desire to preserve personal attachment and avoid sacrificing cognitive competence and resource investment. Additionally, we found that NFI has a significant moderating role in this relationship, highlighting the critical role of user characteristics in shaping the effect of commitment to the status quo in the post-adoption stage.

6.2. Theoretical contributions

This work makes several contributions to the literature. First, our research is founded on temporal construal theory, a novel theoretical lens for understanding IT usage behavior from a temporal distance perspective. Previous research in the IS field has predominantly explored factors such as user demographics, motivations, social influence, and technological benefits to elucidate continued IS usage intention, primarily from a static standpoint [33]. However, growing calls within the technology adoption literature have demanded alternative theoretical explanations, especially those invoking a temporal lens (e.g., [71]). Our study has responded to this call by investigating IS continued usage intention and behavior through the lens of value appreciation grounded in temporal distance. We found that when making decisions for the future, users tend to prioritize primary and goal-related factors (e.g., perceived intelligence of the AI Teaching Assistant), with secondary and goal-irrelevant factors (e.g., perceived empathy of the AI Teaching Assistant) holding more sway in evaluations of the status quo [[38], [66]]. Consequently, our research contributes to the technology usage literature by providing insight into this relatively novel theoretical framework.
Second, our research introduces the novel concepts of perceived status quo usage value and perceived future usage value as criteria for users to assess their continued usage intentions and behavior. These two constructs, rooted in temporal distance assessments, elucidate how users shape their decisions concerning ongoing usage. Notably, many previous studies have treated these assessments interchangeably, overlooking the unique role played by perceived future usage value. This oversight can mislead practitioners who aim to incentivize post-adoption behavior through strategies primarily designed for the initial adoption phase. As such, introducing these IS-specific constructs related to time represents a valuable contribution to the expansion of the technology adoption and perceived value literature.
Third, our research not only contributes to a deeper understanding of continued usage intentions but also broadens the scope of temporal construal literature by integrating NFI into a comprehensive model. In the context of AI teaching assistant usage, the concept of construal fit becomes pivotal. By examining variations in user evaluations of future usage value based on individual characteristics, we extend the temporal construal theory. Essentially, whether an attribute is considered a high-level or low-level construal is not a fixed aspect of an event. Instead, it depends on the alignment between user goals (e.g., using AI for learning or for entertainment) and the technology's attributes.
Fourth, contributing to the IS post-adoption literature, this study shows that once an expectation is confirmed during the post-adoption stage, users are likely to form a sense of commitment to the status quo IS usage. In turn, this commitment significantly influences their ongoing usage behavior. Existing studies have focused primarily on the impact of expectancy confirmation on user satisfaction with technology use [[28], [61]]. However, our study has identified an additional mechanism that elucidates the positive effects of expectation confirmation, illustrating that when their expectations are confirmed, users are more likely to establish a deep connection and commitment to their existing relationship with the technology. This form of commitment has yet to receive adequate attention from the technology adoption literature. Furthermore, our study extends the post-adoption literature by highlighting the crucial role of NFI as a moderator in the relationship between commitment to the status quo and continued usage behavior in the AI chatbot usage context.

6.3. Practical contributions

The findings of the current research also have managerial implications for practitioners. AI chatbots offer advantages including rapid information retrieval, continuous availability, and personalized learning experiences. This potential to replace human agents in the future underscores the need to grasp student motivations and requirements. Despite the substantial research on chatbots, a broader educational perspective is often lacking.
Our results emphasize that user assessments of the current and future usage values of AI chatbots hinge heavily on chatbot IQ and chatbot EQ. IQ, the chatbot's core functionality, significantly enhances user perceptions of current and future value, promoting sustained usage intention. These results imply that e-learning systems incorporating AI chatbots must deliver relevant, reliable, personalized, and up-to-date information in a user-friendly format. If users cannot obtain the desired information, they may find the chatbot unhelpful, which negatively affects their perceptions and discourages continued usage. The intelligence of the AI chatbot is critical for user assessments of its future usage value and consequent continued usage intention. Meanwhile, AI chatbots should not be wholly utilitarian; EQ significantly influences user behavior, particularly during the adoption stage. Empathetic responses and understanding of user experiences and feelings contribute to positive assessments of current usage value. When designing chatbot systems, developers should prioritize the chatbot's EQ and recognize the varying roles it plays at different implementation stages.
Next, NFI exerts a powerful influence on the perception of the value of the AI chatbot, which ultimately influences continued usage behavior. Thus, when considering using AI to replace real humans, practitioners must consider NFI variation and tailor strategies accordingly. Low-NFI users emphasize chatbot functionality––they treat it as a “learning tool” and may stop using it if it fails to answer questions correctly. High-NFI users are more tolerant of chatbot service issues, transitioning emotional attachment to machines with high EQ. Empathy is crucial for capturing high-NFI users. As technology advances, AI chatbots such as ChatGPT demonstrate proficiency at completing education-related tasks. However, doing so may not suffice to replace human teaching assistants due to the interactive nature of teaching and learning. Future AI teaching assistants must integrate emotion recognition and responsive features to emulate human interaction, particularly for high-NFI students.
Finally, this study underscores the significance of confirming expectations and fostering commitment to the AI chatbot during the post-adoption phase. Except for initial continued usage intention, formed during the adoption stage, the evolving experiences of users and their attitudes toward the technology become influential over time in terms of sustaining continued usage. Thus, to validate customer expectations regarding AI chatbots, designers and practitioners must consistently enhance their provision of high-quality service, particularly focusing on IQ, to meet evolving customer expectations and maintain positive attitudes toward the technology.

7. Limitations and future research

Despite the promise of these findings, they should be interpreted within several limitations. First, the sample size is relatively small, albeit acceptable in the general view. Although we adopted PLS-SEM because it is believed to be sufficiently robust in the case of a small sample size, a larger sample size would be desirable to increase the statistical power of the study.
Second, the observation interval between the adoption stage and the post-adoption stage was relatively short (one month). We encourage future studies to examine these relationships over longer time frames to observe more dynamic changes.
Third, this study focuses specifically on the continued usage intention and behavior of students. However, before continued usage intention, it is critical to understand initial adoption intention and behavior. Hence, future research should probe the motivations for non-users’ initial adoption intentions and draw comparisons among the motivational distinctions between adoption and continued usage in the context of AI teaching assistants.
Finally, our study has recognized that although encouraging the continued usage of AI teaching assistants is paramount, doing so introduces complexities around assessing the quality of student learning. Historically, students were assessed by the papers they produced. However, advances in technology have created the potential for AI teaching assistants to complete coursework in ways that are difficult to distinguish from human work, creating evaluation challenges. To ensure the seamless integration of AI teaching assistants into higher education, it is imperative that future research address these challenges. Solutions may involve exploring authentic assessment methods and developing mechanisms for differentiating between robot-generated and human-authored texts.

8. Conclusions

This study has advanced the understanding of AI chatbot continued usage behavior from a temporal distance perspective. We have presented a holistic model based on temporal construal theory to understand the continued usage of AI chatbots. In the adoption stage, we found that users’ continued usage intentions are shaped by assessments of status quo usage value and future usage value, which are influenced by various attributes of AI. User NFI significantly moderates users’ perceptions of the technology, especially when temporal aspects are considered. In the post-adoption stage, we found that—beyond continued usage intention—expectation confirmation and commitment to the status quo critically determine a user's continued usage behavior. This study provides a new perspective on the continued usage of AI chatbots in the higher education context and has managerial implications for institutions and universities that are considering implementing AI solutions to complement the work of teachers.

Funding

This research was funded by the National Natural Science Foundation of China (72272022) and National Natural Science Foundation of China (72304057).

CRediT authorship contribution statement

Hongying Zhao: Writing – original draft, Methodology, Funding acquisition, Conceptualization. Qingfei Min: Writing – review & editing, Supervision, Funding acquisition, Data curation, Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix

Table A1. Measurement items.

ConstructsItemsSources
Perceived empathy (PEM)PEM1: Teaching Assistant XiaoJun is sympathetic.Meyer-Waarden et al. [41], Pelau et al. [50]
PEM2: Teaching Assistant XiaoJun is capable of understanding my needs.
PEM3: Teaching Assistant XiaoJun is capable of understanding my feelings.
Perceived intelligence (PI)PI1: Teaching Assistant XiaoJun understands my questions and can provide relevant answers.Chang-Chien et al. [7]
PI2: Teaching Assistant XiaoJun provides context-relevant information that is suited to my needs.
PI3: Teaching Assistant XiaoJun can answer questions in the professional field.
Need for human interaction (NFI)NFI1: I enjoy the process of communicating with the human teaching assistant more than communicating with Teaching Assistant XiaoJun.Song et al. [60],
Dabholkar and Bagozzi [12]
NFI2: It is enjoyable for me to communicate directly with a human teaching assistant during the learning process.
NFI3: It bothers me to use a machine (e.g., Teaching Assistant XiaoJun) when I could talk to the human teaching assistant instead.
Perceived status quo usage value (PSQV)PSQV1: At present, Teaching Assistant XiaoJun makes me feel comfortable.Goyal et al. [20]
PSQV2: At present, Teaching Assistant XiaoJun makes me feel free of uncertainty.
PSQV3: At present, Teaching Assistant XiaoJun is beneficial to my routine activities.
Perceived future usage value (PFUV)PFUV1: In the future, Teaching Assistant XiaoJun will make me feel comfortable.
PFUV2: In the future, Teaching Assistant XiaoJun will make me feel free of uncertainty.
PFUV3: In the future, Teaching Assistant XiaoJun will be beneficial to my routine activities.
Expectation confirmation (EC)Expectation confirmation was calculated by subtracting the perceived future usage value (collected at T1, representing expectations) from the perceived status quo usage value (collected at T2, reflecting performance) for each item.
EC1=PSQV1 (collected at T2) – PFUV1 (collected at T1)
EC2=PSQV2 (collected at T2) – PFUV2 (collected at T1)
EC3=PSQV3 (collected at T2) – PFUV3 (collected at T1)
EC4=PSQV4 (collected at T2) – PFUV4 (collected at T1)
Prakash and Lounsbury [54]
Commitment to status quo (CTSQ)CTSQ1: At present, I have a strong personal attachment to Teaching Assistant XiaoJun.Goyal et al. [20]
CTSQ2: Currently, continued use of Teaching Assistant XiaoJun matches my needs.
CTSQ3: Too many things would be disrupted if I discontinued using Teaching Assistant XiaoJun.
Continued usage intention (CUI)CUI1: I intend to continue using Teaching Assistant XiaoJun.Goyal et al. [20]
CUI2: My intention is to continue using Teaching Assistant XiaoJun.
CUI3: I expect to continue using Teaching Assistant XiaoJun in the future.
Continued usage behaviorFrequency: How often do you use Teaching Assistant XiaoJun?
1. 1–3 times a month 2. Once a week 3. Once a day 4. More than once a day
Venkatesh et al. [70]

References

Cited by (0)

Hongying Zhao is an Assistant Professor in the School of Economics and Management at the Dalian University of Technology. Her research interests centered on understanding users’ behavior in the online community from a dynamic viewpoint so as to inspires practitioners on means to obtain sustainable success across different stages. She received her Ph.D. in Information Systems from City University of Hong Kong. Her email address is hongyzhao@dlut.edu.cn.
Qingfei Min is Professor of Information Systems at School of Economics and Management, Dalian University of Technology, China. His research interests include IT/IS implementation and adoption, ecommerce/m-commerce behavior and strategies, global virtual team, and social media. He received his Ph.D. in Information Systems from Dalian University of Technology. He has published several studies in Information Systems Journal, Information and Management, International Journal of Mobile Commerce, Computers in Human Behavior as well as in Chinese academic journals and international conferences. Qingfei Min is the corresponding author and can be contacted at: minqf@dlut.edu.cn.
View Abstract