1 Introduction
1简介

Generative artificial intelligence (AI) language models (LMs) such as OpenAI’s GPT-2, GPT-3 and GPT-4 have captivated educators’ interest, because they can generate large chunks of coherent text indistinguishable from human writing (Brown et al., 2020) and proficiently perform a variety of natural language processing tasks when instructed or prompted (Ouyang et al., 2022). Furthermore, ChatGPT has popularized interaction with LMs through a chatbot interface, that is, a conversational user interface that enables people to engage in meaningful verbal or text-based exchanges with an LM (Kim et al., 2022). As ChatGPT has captured popular imagination, ChatGPT is used as a catchall phrase for chatbots that use transformers-based LMs (Vaswani et al., 2017).
OpenAI 的 GPT-2、GPT-3 和 GPT-4 等生成式人工智能 (AI) 语言模型 (LM) 引起了教育工作者的兴趣，因为它们可以生成与人类书写无法区分的大块连贯文本（Brown 等人， 2020 ）并在指示或提示时熟练地执行各种自然语言处理任务（Ouyang et al., 2022 ）。此外，ChatGPT 通过聊天机器人界面普及了与 LM 的交互，即对话式用户界面，使人们能够与 LM 进行有意义的口头或基于文本的交流（Kim 等人， 2022 ）。由于 ChatGPT 吸引了大众的想象力，ChatGPT 被用作使用基于 Transformer 的 LM 的聊天机器人的总称（Vaswani 等人， 2017 ）。

ChatGPT enables students to write with a machine-in-the-loop, which refers to a collaborative process between a student and a chatbot to complete a writing task. As defined by Clark et al. (2018) and illustrated in Fig. 1, the process is iterative. First, a student prompts or delivers a set of instructions to guide ChatGPT such as a question, an imperative statement or an excerpt from a text. Based on its understanding of the student’s prompt, ChatGPT generates output. The student then evaluates the output, accepting, rejecting or modifying ChatGPT’s output for integration into the student’s written composition. The cycle loops until the completion of the writing task with the student retaining full control over the written composition. Having previously been applied to creative writing, researchers found writers appreciated ChatGPT’s fresh ideas, which helped overcome writer's block, while still maintaining ownership over their work. However, the quality of written compositions have not necessarily improved with ChatGPT suggestions (Calderwood et al., 2020; Clark et al., 2018).
ChatGPT 使学生能够使用机器在环进行写作，这是指学生和聊天机器人之间完成写作任务的协作过程。正如克拉克等人所定义的。 ( 2018 ) 并如图1所示，该过程是迭代的。首先，学生提示或提供一组指导 ChatGPT 的指令，例如问题、命令式陈述或文本摘录。 ChatGPT 根据对学生提示的理解生成输出。然后，学生评估输出，接受、拒绝或修改 ChatGPT 的输出，以便整合到学生的书面作文中。这个循环不断循环，直到完成写作任务，学生保留对书面作文的完全控制。研究人员发现，此前曾应用于创意写作，作家们很欣赏 ChatGPT 的新鲜想法，这有助于克服作家的障碍，同时仍然保持对自己作品的所有权。然而，ChatGPT 建议并不一定会提高书面作文的质量（Calderwood 等人， 2020 ；Clark 等人， 2018 ）。

Fig. 1 图1
figure 1

Machine-in-the-loop system characteristics with arrows indicating relationships between a student, ChatGPT, and ChatGPT output to complete student writing
机器在环系统特征，箭头指示学生、ChatGPT 和 ChatGPT 输出之间的关系，以完成学生写作

Full size image 全尺寸图像

Notwithstanding ChatGPT’s potential benefits, the integration of ChatGPT into the English as a foreign language (EFL) writing classroom remains largely unexplored in terms of students' experiences and perceptions. This study aims to fill this research gap in the context of Hong Kong secondary school students learning to compose a written EFL composition with ChatGPT support. The objective is to explore how students perceive their experience of learning this innovative writing approach in terms of their motivation to learn about ChatGPT, cognitive load, and satisfaction with the learning process. These aspects are critical as they directly impact students' learning behaviors, engagement, and ultimately, their writing outcomes. Furthermore, understanding these aspects can provide valuable insights for educators, informing instructional approaches for integrating ChatGPT into the EFL writing classroom. The overarching question guiding this research is: How do EFL students perceive learning to write with ChatGPT in a classroom context?
尽管 ChatGPT 具有潜在的好处，但就学生的体验和看法而言，将 ChatGPT 融入英语作为外语 (EFL) 写作课堂的情况在很大程度上仍未得到探索。本研究旨在填补香港中学生在 ChatGPT 支持下学习撰写 EFL 书面作文的研究空白。目的是探索学生如何看待他们学习这种创新写作方法的经历，包括他们学习 ChatGPT 的动机、认知负荷和对学习过程的满意度。这些方面至关重要，因为它们直接影响学生的学习行为、参与度，并最终影响他们的写作成果。此外，了解这些方面可以为教育工作者提供宝贵的见解，为将 ChatGPT 融入 EFL 写作课堂的教学方法提供信息。指导这项研究的首要问题是：EFL 学生如何看待在课堂环境中使用 ChatGPT 学习写作？

1.1 Potential of ChatGPT in the EFL writing classroom
1.1 ChatGPT 在 EFL 写作课堂中的潜力

In the EFL writing classroom, students can face difficulty in retrieving intended English words and take time to translate ideas from their first language to English (Gayed et al., 2022). Students can struggle to generate ideas independently (Woo et al., 2023) and may not have sufficient and effective engagement with peer feedback in the writing process (Zhang & Hyland, 2023), although collaborative writing is an effective pedagogical practice (Li & Zhang, 2023) and students’ quality of writing can benefit from collaboration (Hsu, 2023).
在 EFL 写作课堂上，学生在检索想要的英语单词时可能会遇到困难，并且需要时间将想法从母语翻译成英语（Gayed 等人， 2022 ）。尽管协作写作是一种有效的教学实践（Li＆Zhang），但学生可能很难独立产生想法（Woo等人， 2023 ），并且在写作过程中可能无法充分有效地参与同伴反馈（Zhang＆Hyland， 2023 ） ， 2023 ）和学生的写作质量可以从合作中受益（Hsu， 2023 ）。

The implementation of ChatGPT in an EFL writing classroom may support learning opportunities for students. This is because a chatbot can act as an ideal collaborative partner for EFL students (Guo et al., 2022), and ChatGPT is highly capable of natural language tasks such as brainstorming ideas, generating texts, answering questions, rewriting texts and summarizing texts (Ouyang et al., 2022). Conceptual studies have explored the use of ChatGPT in EFL writing classrooms, suggesting hypothetical use cases. For instance, Hwang and Chen (2023) suggested the potential application of students using ChatGPT as a proofreader for academic writing in EFL courses. Su et al. (2023) explored the potential of ChatGPT in assisting students with preparing outlines, revising content, proofreading, and reflecting.
在 EFL 写作课堂上实施 ChatGPT 可以为学生提供学习机会。这是因为聊天机器人可以充当 EFL 学生理想的协作伙伴（Guo et al., 2022 ），并且 ChatGPT 非常有能力执行自然语言任务，例如集思广益、生成文本、回答问题、重写文本和总结文本（欧阳等人， 2022 ）。概念研究探索了 ChatGPT 在 EFL 写作课堂中的使用，提出了假设的用例。例如，Hwang 和 Chen ( 2023 ) 建议学生使用 ChatGPT 作为 EFL 课程中学术写作的校对者的潜在应用。苏等人。 ( 2023 ) 探索了 ChatGPT 在协助学生准备大纲、修改内容、校对和反思方面的潜力。

However, some EFL teachers fear that students may become dependent on ChatGPT and its dubious suggestions (Ulla et al., 2023). ChatGPT could reinforce biased ideas (Mohamed, 2023). Additionally, students could use ChatGPT with neither much effort nor student input to complete writing assignments, undermining students’ acquisition of English and writing skills (Gayed et al., 2022), and critical and creative thinking (Barrot, 2023). Empirical studies featuring actual use cases of ChatGPT support in the EFL writing classroom show mixed results. For instance, Cao and Zhong (2023) compared ChatGPT feedback, EFL teacher feedback and student feedback for improving 45 university students’ written translation performance and found ChatGPT feedback was less effective than other feedback types in improving performance. On the other hand, Athanassopoulos et al. (2023) examined ChatGPT’s effectiveness as a writing vocabulary and grammar feedback tool for eight, 15-year old migrants and refugees. After writing a task and receiving improved versions of their writing generated by ChatGPT, the students could increase the total number of words, the unique words and the number of words per sentence when writing a similar task.
然而，一些 EFL 教师担心学生可能会依赖 ChatGPT 及其可疑的建议（Ulla 等人， 2023 ）。 ChatGPT 可能会强化有偏见的想法（Mohamed， 2023 ）。此外，学生可以使用 ChatGPT 来完成写作作业，既不需要付出太多努力，也不需要学生投入，这会损害学生对英语和写作技能的掌握（Gayed 等， 2022 ）以及批判性和创造性思维（Barrot， 2023 ）。针对 EFL 写作课堂中 ChatGPT 支持的实际用例进行的实证研究显示出不同的结果。例如，Cao和Zhong（ 2023 ）比较了ChatGPT反馈、EFL教师反馈和学生反馈以提高45名大学生的书面翻译成绩，发现ChatGPT反馈在提高成绩方面不如其他反馈类型有效。另一方面，Athanassopoulos 等人。 ( 2023 ) 检查了 ChatGPT 作为 8 岁、15 岁移民和难民的写作词汇和语法反馈工具的有效性。在编写任务并收到 ChatGPT 生成的改进版本后，学生在编写类似任务时可以增加总单词数、唯一单词和每个句子的单词数。

1.2 Genre writing and prompt engineering as genre in the EFL classroom
1.2流派写作和提示工程作为英语课堂中的流派

How teachers should approach the instruction of writing with ChatGPT in an EFL classroom is a complex issue. From an EFL teaching and learning perspective, a teacher adopting an explicit, instructional approach to EFL writing appears necessary for whether implementing ChatGPT intentionally benefits or hinders students’ acquisition of knowledge and skills. In this regard, although process writing has been a popular, inductive writing strategy, Hyland (2007) has argued it has limited value for EFL learners who lack access to cultural knowledge that facilitates effective, independent writing. Instead, this study approaches EFL students’ acquisition of writing through genre, which emphasizes communicating effectively through different types of texts, and their specific conventions, language features, and structures (Hyland, 2019). As illustrated in Fig. 2, a genre-approach to writing instruction is explicit, including stages such as a teacher modeling a genre, joint construction of a text in the genre, and a student’s independent construction of a text in the genre. Like a conventional EFL teacher, ChatGPT could support students at each stage by, for example, generating model texts of a genre, identifying the genre’s linguistic features, collaboratively writing sections of a text with a student, and suggesting vocabulary, grammar and outlines and providing feedback for a student’s independent construction of a text.
教师应如何在 EFL 课堂​​上使用 ChatGPT 进行写作教学是一个复杂的问题。从 EFL 教学和学习的角度来看，教师采用明确的 EFL 写作教学方法对于实施 ChatGPT 是否有意有益于或阻碍学生获取知识和技能似乎是必要的。在这方面，尽管过程写作一直是一种流行的归纳写作策略，但 Hyland ( 2007 ) 认为，对于缺乏促进有效、独立写作的文化知识的英语学习者来说，过程写作的价值有限。相反，本研究通过体裁来研究英语FL学生的写作习得，强调通过不同类型的文本及其特定的惯例、语言特征和结构进行有效的沟通（Hyland， 2019 ）。如图2所示，写作教学的体裁方法是明确的，包括教师建模体裁、联合构建该体裁文本以及学生独立构建该体裁文本等阶段。与传统的 EFL 教师一样，ChatGPT 可以在每个阶段为学生提供支持，例如，生成某个体裁的模型文本，识别该体裁的语言特征，与学生协作编写文本的各个部分，以及建议词汇、语法和大纲，并提供对学生独立构建文本的反馈。

Fig. 2 图2
figure 2

Three sequential stages of a genre approach to writing and ChatGPT’s support at each stage
体裁写作方法的三个连续阶段以及每个阶段 ChatGPT 的支持

Full size image 全尺寸图像

When students write with a machine-in-the-loop, the effect of ChatGPT on students' knowledge and skill development in genre writing depends on how well students give instructions or prompts for ChatGPT. Proficient crafting of prompts or prompt engineering can significantly enhance the quality of ChatGPT’s generated output and the overall effectiveness of the interaction with ChatGPT (Reynolds & McDonell, 2021). Since constructing appropriate prompts is not straightforward for non-technical users (Zamfirescu-Pereira et al., 2023), and ChatGPT prompts are an emergent genre, scholars have proposed example prompts for hypothetical ChatGPT use cases in the literature (Hwang & Chen, 2023; Kohnke et al., 2023; Su et al., 2023).
当学生使用机器在环进行写作时，ChatGPT 对学生体裁写作知识和技能发展的影响取决于学生对 ChatGPT 的指导或提示的程度。熟练地制作提示或提示工程可以显着提高 ChatGPT 生成输出的质量以及与 ChatGPT 交互的整体有效性（Reynolds & McDonell， 2021 ）。由于构建适当的提示对于非技术用户来说并不简单（Zamfirescu-Pereira 等人， 2023 ），并且 ChatGPT 提示是一种新兴类型，因此学者们在文献中为假设的 ChatGPT 用例提出了示例提示（Hwang & Chen， 2023） ；Kohnke 等人， 2023 ；Su 等人， 2023 ）。

The implication for the EFL writing classroom is that ChatGPT’s capability to support students at different stages of genre writing would depend on teachers not only developing students’ knowledge and skills of the target text type but also developing student’s prompt engineering knowledge and skills. Furthermore, the authors anticipate prompt engineering instruction could compose a significant part of students’ learning to write with ChatGPT. For instance, teachers could orient students towards what AI is, what a chatbot is, ChatGPT capabilities, exemplary prompts to unlock ChatGPT capabilities and vocabulary and grammar for students to independently construct prompts to unlock ChatGPT capabilities. Given the iterative nature of the machine-in-the-loop writing process, students’ may spend much time crafting prompts. Thus, exploring EFL students' perceptions during the prompt engineering phase of genre writing could inform more effective instruction to develop students’ prompt engineering knowledge and skills.
对 EFL 写作课堂的启示是，ChatGPT 支持处于不同体裁写作阶段的学生的能力不仅取决于教师培养学生目标文本类型的知识和技能，还取决于培养学生即时的工程知识和技能。此外，作者预计及时的工程指导可以成为学生学习使用 ChatGPT 进行写作的重要组成部分。例如，教师可以引导学生了解什么是人工智能、什么是聊天机器人、ChatGPT 功能、解锁 ChatGPT 功能的示例提示以及词汇和语法，以便学生独立构建解锁 ChatGPT 功能的提示。考虑到机器在环写作过程的迭代性质，学生可能会花费大量时间来制作提示。因此，探索 EFL 学生在体裁写作的即时工程阶段的看法可以为更有效的指导提供帮助，以培养学生的即时工程知识和技能。

In summary, previous research has suggested potential for ChatGPT to support EFL students’ writing yet realizing that potential in the classroom may require not only effective writing instruction but also effective prompt engineering instruction. Furthermore, ChatGPT may convey benefits for EFL students’ genre writing if ChatGPT does not replace the teacher, but rather students use ChatGPT alongside teacher instruction to ethically and effectively develop writing skills (Shaikh et al., 2023). Empirical research on EFL student perceptions is a means to evaluate student experiences when learning to write with ChatGPT a classroom context.
总之，之前的研究表明 ChatGPT 支持 EFL 学生写作的潜力，但意识到课堂上的潜力可能不仅需要有效的写作指导，还需要有效的即时工程指导。此外，如果 ChatGPT 不取代教师，而是学生在教师指导的同时使用 ChatGPT 以合乎道德且有效地培养写作技能，那么 ChatGPT 可能会给 EFL 学生的体裁写作带来好处（Shaikh 等人， 2023 ）。对 EFL 学生认知的实证研究是评估学生在课堂环境中使用 ChatGPT 学习写作时的体验的一种手段。

1.3 Student perceptions about learning to write with ChatGPT
1.3学生对使用 ChatGPT 学习写作的看法

While ChatGPT shows potential to support students’ writing, it is crucial to understand students’ perceptions about learning to write with ChatGPT in their classroom context. Student perceptions encompass students' subjective assessment of their learning environment (e.g. curriculum; instructional methods and materials; and other services and contextual factors) (Biggs, 1999), and can significantly influence their learning behaviors, engagement, and ultimately academic achievement. This is because positive perceptions may foster a deep learning approach, whereas negative perceptions may facilitate a surface learning approach. The following literature review elaborates three aspects of student perceptions that are often examined to evaluate learning environments.
虽然 ChatGPT 显示出支持学生写作的潜力，但了解学生对在课堂环境中使用 ChatGPT 学习写作的看法至关重要。学生感知包括学生对其学习环境的主观评估（例如课程、教学方法和材料以及其他服务和背景因素）（Biggs， 1999 ），并且可以显着影响他们的学习行为、参与度以及最终的学业成绩。这是因为积极的看法可能会促进深度学习方法，而消极的看法可能会促进表面学习方法。以下文献综述详细阐述了学生认知的三个方面，这些方面经常被用来评估学习环境。

1.3.1 Motivation to learn
1.3.1学习动机

Motivation to learn refers to students' desire and willingness to engage with the learning materials and activities (Keller, 1987). It is an important factor influencing how students approach and persist with learning tasks. It's especially crucial in the context of EFL writing, a challenging task demanding cognitive effort and continual practice. In the case of writing with ChatGPT, students' motivation can be influenced by their perceived usefulness and ease of use of the technology (Davis, 1989). Motivation to learn is often evaluated through questionnaire items. Hwang and Chang (2011) found their formative assessment-based mobile learning environment improved students’ learning motivation toward the target content, the authors suggesting appropriate challenges had motivated students during the learning process. Shim et al. (2023) found their experiential chatbot workshop was instrumental in positively motivating their students to learn chatbot competencies. Kim and Lee (2023) found socio-economically disadvantaged Korean middle school students were far more motivated to learn about AI than students who were not socio-economically disadvantaged. On the other hand, Hwang et al. (2013) found a concept map-embedded game did not have a significant impact on students’ learning motivation when compared to a digital game without a concept mapping strategy. Alternatively, Jeon (2022) adopted qualitative methods to explore how chatbots affected EFL primary students’ motivation to learn English, identifying chatbot affordances and limitations that facilitated and decreased, respectively, students’ motivation to learn English through chatbots. Similarly, Chan and Hu (2023) asked Hong Kong university students open-ended questions to collect data on students’ willingness to use ChatGPT and found most participants were motivated to use it, identifying several reasons.
学习动机是指学生参与学习材料和活动的愿望和意愿（Keller， 1987 ）。它是影响学生如何完成和坚持学习任务的重要因素。这对于英语写作来说尤其重要，这是一项具有挑战性的任务，需要认知努力和持续练习。在使用 ChatGPT 进行写作的情况下，学生的动机可能会受到他们感知到的技术有用性和易用性的影响（Davis， 1989 ）。学习动机通常通过问卷项目来评估。 Hwang 和 Chang ( 2011 ) 发现他们的基于形成性评估的移动学习环境提高了学生对目标内容的学习动机，作者认为适当的挑战在学习过程中激发了学生的积极性。希姆等人。 ( 2023 ) 发现他们的体验式聊天机器人研讨会有助于积极激励学生学习聊天机器人能力。 Kim 和 Lee ( 2023 ) 发现，与社会经济状况良好的学生相比，社会经济地位处于弱势的韩国中学生学习人工智能的积极性要高得多。另一方面，黄等人。 ( 2013 )发现，与没有概念图策略的数字游戏相比，嵌入概念图的游戏对学生的学习动机没有显着影响。另外，Jeon ( 2022 ) 采用定性方法来探索聊天机器人如何影响 EFL 小学生学习英语的动机，确定聊天机器人的可供性和限制，分别促进和降低学生通过聊天机器人学习英语的动机。 同样，Chan 和 Hu（ 2023 ）向香港大学生提出了开放式问题，以收集有关学生使用 ChatGPT 意愿的数据，发现大多数参与者都有使用它的动机，并找出了几个原因。

1.3.2 Cognitive load
1.3.2认知负荷

Cognitive load theory (Sweller, 1988) posits that people’s capacity to process information during learning is limited. In this way, a heavy cognitive load impedes learning but a manageable level of cognitive load facilitates it. Furthermore, cognitive load is a multidimensional concept comprising two components (Paas, 1992). Mental load refers to the load imposed by task demands. Mental effort refers to the amount of cognitive capacity allocated to address the task demands. Sweller et al. (1998) elaborated a cognitive architecture and proposed that when designing instruction, information should be organized and presented in a way to reduce cognitive load on working memory and increase knowledge stored in long-term memory. In evaluating innovative educational technology approaches, researchers have evaluated cognitive load in students through surveys and have found, for example, a formative assessment-based mobile learning environment could improve learning achievement with appropriate cognitive load (Hwang & Chang, 2011); and a concept map-embedded game also improved students’ learning achievement and decreased their cognitive load (Hwang et al., 2013).
认知负荷理论（Sweller， 1988 ）认为人们在学习过程中处理信息的能力是有限的。这样，沉重的认知负荷会阻碍学习，但可控水平的认知负荷会促进学习。此外，认知负荷是一个包含两个组成部分的多维概念（Paas， 1992 ）。精神负荷是指任务要求所施加的负荷。脑力劳动是指为满足任务需求而分配的认知能力的量。斯韦勒等人。 ( 1998 )详细阐述了一种认知架构，并提出在设计教学时，信息的组织和呈现方式应减少工作记忆的认知负荷，增加长期记忆中存储的知识。在评估创新教育技术方法时，研究人员通过调查评估了学生的认知负荷，并发现，例如，基于形成性评估的移动学习环境可以通过适当的认知负荷提高学习成绩（Hwang＆Chang， 2011 ）；概念图嵌入游戏也提高了学生的学习成绩并减轻了他们的认知负担（Hwang et al., 2013 ）。

1.3.3 Satisfaction with learning
1.3.3学习满意度

Satisfaction is a basic measure of how participants react to a program or learning process. It can be characterized as either positive or negative. Importantly, although satisfaction with the learning process does not ensure learning, dissatisfaction may impede learning (Kirkpatrick & Kirkpatrick, 2006). When designing instruction, high satisfaction can validate standards of performance for future programs. Satisfaction is often evaluated quantitatively through questionnaires. For instance, Fisher et al. (2010) found that teachers who participated in a virtual professional development program were as satisfied as teachers who participated in an in person program; and that students were satisfied by the instruction from both groups of teachers. Shim et al. (2023) found 91% of their students were satisfied with an experiential learning chatbot workshop with no students indicating dissatisfaction. With regards to ChatGPT, Amaro et al. (2023) found that their cohort of Italian university students exhibited a high level of satisfaction during a guided interaction with ChatGPT. However, they also observed that satisfaction levels decreased because students became aware of ChatGPT’s ability generate false information, particularly when students’ awareness arose early in the interaction. Escalante et al. (2023) conducted a study in which 43 university EFL students received writing feedback from both human tutors and ChatGPT over a six-week period. The students reported similar levels of satisfaction with the feedback from both sources. Alternatively, in a mixed-methods study by Belda-Medina and Calvo-Ferrer (2022), 176 Spanish and Polish undergraduates interacted with three AI chatbots over a four-week period. Through analysis of survey data, the researchers found gender-related differences in levels of satisfaction and by analysis of students’ written reports to open-ended questions, identified key factors for students’ satisfaction.
满意度是衡量参与者对计划或学习过程如何反应的基本指标。它可以被表征为积极的或消极的。重要的是，虽然对学习过程的满意并不能确保学习，但不满意可能会阻碍学习（Kirkpatrick & Kirkpatrick， 2006 ）。在设计教学时，高满意度可以验证未来项目的绩效标准。满意度通常通过问卷调查进行定量评估。例如，费舍尔等人。 （ 2010 ）发现参加虚拟专业发展计划的教师与参加现场计划的教师一样满意；学生对两组教师的教学均感到满意。希姆等人。 ( 2023 ) 发现 91% 的学生对体验式学习聊天机器人研讨会感到满意，没有学生表示不满意。关于 ChatGPT，Amaro 等人。 ( 2023 ) 发现，他们的意大利大学生群体在与 ChatGPT 的引导互动中表现出很高的满意度。然而，他们还观察到，满意度水平下降是因为学生意识到 ChatGPT 产生虚假信息的能力，特别是当学生在互动的早期意识到这一点时。埃斯卡兰特等人。 ( 2023 ) 进行了一项研究，其中 43 名大学 EFL 学生在六周内收到了来自真人导师和 ChatGPT 的写作反馈。学生们对两个来源的反馈的满意度相似。 另外，在 Belda-Medina 和 Calvo-Ferrer 的一项混合方法研究（ 2022 ）中，176 名西班牙和波兰本科生在为期四个星期的时间内与三个人工智能聊天机器人进行了互动。通过分析调查数据，研究人员发现了与性别相关的满意度差异，并通过分析学生对开放式问题的书面报告，确定了学生满意度的关键因素。

To conclude, after the literature review, the overarching research question is operationalized into three questions, each addressing a particular aspect of student perception:
总而言之，在文献综述之后，总体研究问题被分解为三个问题，每个问题都涉及学生感知的一个特定方面：

  • RQ1: How does the use of ChatGPT in writing impact EFL students' motivation to learn about ChatGPT?
    RQ1：ChatGPT 在写作中的使用如何影响 EFL 学生学习 ChatGPT 的动机？

  • RQ2: What is the cognitive load experienced by EFL students when writing with ChatGPT?
    RQ2：EFL 学生在使用 ChatGPT 写作时经历的认知负荷是多少？

  • RQ3: How satisfied are EFL students with the experience of writing with ChatGPT?
    RQ3：EFL 学生对使用 ChatGPT 进行写作的体验有多满意？

2 Methods
2方法

2.1 Context and sample
2.1背景和样本

This research used a convenience sample. Twenty-one students voluntarily participated in the study, where they were provided information about the study's objectives and tasks, their rights as participants, and the option to withdraw their participation at any point during the study. They were informed in English and Chinese language, verbally and in text, and were allowed to raise any questions or concerns about their participation with the researchers. No students declined participation.
本研究使用了方便样本。 21 名学生自愿参与了这项研究，他们获得了有关研究的目标和任务、他们作为参与者的权利以及在研究期间随时退出参与的选择的信息。他们以英语和中文、口头和文字形式被告知，并被允许向研究人员提出有关他们参与的任何问题或疑虑。没有学生拒绝参与。

The participants in this study were students from an all-girls secondary school in Hong Kong where the first author worked as an English as a Foreign Language (EFL) teacher. The school’s students have academic achievement ranging from the 44th to the 55th percentile based on their results in the secondary school entrance exams (Lee & Chiu, 2017) and compared to peers in the school’s geographic district. The demographic information of the sample is shown in Table 1. The average age was 13.6 years. Seven students lived in public housing, indicating a lower socio-economic status background in Hong Kong. Student’s EFL writing proficiency was defined by their last EFL writing exam mark. As the school’s passing mark for a writing exam is 50 out of 100, the majority of students (n = 11) were mediocre writers scoring between 40 and 60.
这项研究的参与者是来自香港一所女子中学的学生，第一作者是该校的英语作为外语（EFL）教师。根据中学入学考试的成绩（Lee & Chiu， 2017 ）并与学校所在地理区域的同龄人进行比较，学校学生的学业成绩在第 44 到 55 个百分点之间。样本人口统计信息如表1所示。平均年龄为13.6岁。七名学生住在公屋，显示香港社会经济地位背景较低。学生的 EFL 写作水平由他们上次的 EFL 写作考试成绩决定。由于学校写作考试的及格分数为 50 分（满分 100 分），因此大多数学生（ n = 11）都是得分在 40 到 60 之间的平庸作家。

Table 1 Participants’ demographic information
表1 参与者的人口统计信息
Full size table 全尺寸桌子

60% of students (n = 12) reported having used ChatGPT prior to the workshop, and 40% (n = 8) reported not having used ChatGPT. However, only 25% of students (n = 5) reported that they had used ChatGPT to complete English language homework, suggesting the majority of students have no experience with ChatGPT use cases in the EFL writing classroom.
60% 的学生 ( n = 12) 表示在研讨会之前使用过 ChatGPT，40% ( n = 8) 表示未使用 ChatGPT。然而，只有 25% 的学生 ( n = 5) 表示他们曾使用 ChatGPT 来完成英语作业，这表明大多数学生没有在 EFL 写作课堂上使用 ChatGPT 的经验。

2.2 Materials and procedure
2.2材料和程序

The study took place in the school’s STEM classroom on July 5, 2023 and repeated on July 6. Six students attended on July 5 and 16 on July 6. The study’s environment for learning to write with ChatGPT took the form of a human-AI creative writing workshop. Each workshop lasted one-hour, 45-min. Because writing with ChatGPT in the EFL classroom is novel, the authors’ developed the workshop activities and materials by design-based research (DBR) (Wang & Hannafin, 2005), that is, a flexible and systematic methodology that can improve educational practice iteratively through design, development, implementation and analysis. The authors adopted an outcome-based learning design, that is, a framework for describing learning environments and learning activities (Conole & Wills, 2013). First, the researchers designed the workshop’s purpose and intended learning outcomes (ILOs), that is, what students should achieve by the end of the workshop. Then the authors designed the learning activities, that is, basic units of interaction with or among learners. Table 2 summarizes the workshop design, which comprises its (1) title, (2) purpose, (3) ILOs, (4) learning activities, and (5) materials and resources. By evaluating student perceptions, the learning design can be improved for subsequent implementations.
该研究于 2023 年 7 月 5 日在学校的 STEM 教室进行，并于 7 月 6 日重复进行。六名学生于 7 月 5 日和 7 月 6 日参加。该研究使用 ChatGPT 学习写作的环境采用了人类人工智能创意的形式写作研讨会。每个研讨会持续一小时 45 分钟。由于在 EFL 课堂​​上使用 ChatGPT 进行写作是新颖的，作者通过基于设计的研究 (DBR)（Wang & Hannafin， 2005 ）开发了研讨会活动和材料，即一种灵活且系统的方法，可以迭代地改进教育实践通过设计、开发、实施和分析。作者采用了基于结果的学习设计，即描述学习环境和学习活动的框架（Conole & Wills， 2013 ）。首先，研究人员设计了研讨会的目的和预期学习成果（ILO），即学生在研讨会结束时应该实现的目标。然后作者设计了学习活动，即与学习者或学习者之间互动的基本单位。表2总结了研讨会的设计，包括 (1) 标题、(2) 目的、(3) ILO、(4) 学习活动以及 (5) 材料和资源。通过评估学生的看法，可以改进学习设计以供后续实施。

Table 2 Workshop learning design
表2 工作坊学习设计
Full size table 全尺寸桌子

In the workshop, the authors introduced students to the genre of effective written communication with chatbots before introducing students to the writing task that they would attempt with ChatGPT. (1) The concept of chatbots was introduced using an inductive approach by showing a chatbot screenshot and asking students, “What are you looking at?” (2) Students were asked to interact with a chatbot, before asking students what this type of generative AI is and how to interact with it. (3) The features of chatbots were introduced, including turn-taking and memory. (4) The principles for chatbot prompting such as the garbage-in-garbage-out principle were introduced by showing a chatbot screenshot to students and asking, “What is a problem with this conversation?” (5) For students to take advantage of ChatGPT’s novel capabilities and to get desired output, the concepts of prompts and prompt engineering were defined. The authors introduced different ChatGPT use cases for writing and example prompts for those use cases based on classmates’ actual prompts and theoretical prompts from a literature review. The use cases included asking ChatGPT to act as a particular role, to act as a search engine, to analyze a text input, to answer a question, to auto-complete a text input, to explain its reasoning for its text output, to paraphrase a text input, to provide additional information to its text output, to summarize a text input, and to translate a text input. The authors did not introduce a use case of prompting ChatGPT to generate a complete composition that replaces human effort in writing. The instructional materials such as the slide deck (see Supplemental Material) were delivered in English by the first author. At the same time, the first author’s colleague provided simultaneous spoken translation in Cantonese Chinese language.
在研讨会上，作者向学生介绍了与聊天机器人进行有效书面交流的类型，然后向学生介绍了他们将使用 ChatGPT 尝试的写作任务。 (1) 通过展示聊天机器人屏幕截图并询问学生“你在看什么？”，使用归纳法引入了聊天机器人的概念。 (2) 学生被要求与聊天机器人进行交互，然后再询问学生这种类型的生成人工智能是什么以及如何与其交互。 (3)介绍了聊天机器人的特点，包括轮流和记忆。 （4）通过向学生展示聊天机器人截图并询问“这次对话有什么问题吗？”，介绍了垃圾进垃圾出原则等聊天机器人提示原理。 (5) 为了让学生利用 ChatGPT 的新颖功能并获得所需的输出，定义了提示和提示工程的概念。作者根据同学的实际提示和文献综述中的理论提示，介绍了不同的 ChatGPT 写作用例以及这些用例的示例提示。用例包括要求 ChatGPT 充当特定角色、充当搜索引擎、分析文本输入、回答问题、自动完成文本输入、解释其文本输出的推理、释义文本输入，为其文本输出提供附加信息，总结文本输入，以及翻译文本输入。作者没有介绍提示 ChatGPT 生成完整作文来代替人类写作的用例。幻灯片等教学材料（参见补充材料）由第一作者用英语提供。 同时，第一作者的同事提供了粤语同声翻译。

After the introduction to prompt engineering for ChatGPT, students began writing with ChatGPT and other state-of-the-art chatbots on school-supplied iPads, on which the Platform for Open Exploration (POE) app was loaded. At the time of study, the app granted free access to ChatGPT and five other chatbots (i.e. Sage, GPT-4, Claude + , Claude-instant, and Google-PaLM) that rely on commercial LMs hundreds of billions of parameters in size. Figure 3 shows the POE app interface on iPad from which students could select from the six chatbots.
在介绍了 ChatGPT 的提示工程后，学生们开始在学校提供的 iPad 上使用 ChatGPT 和其他最先进的聊天机器人进行写作，iPad 上加载了开放探索平台 (POE) 应用程序。在研究时，该应用程序允许免费访问 ChatGPT 和其他五个聊天机器人（即 Sage、GPT-4、Claude +、Claude-instant 和 Google-PaLM），这些机器人依赖于商业 LM 的数千亿个参数。图3显示了 iPad 上的 POE 应用程序界面，学生可以从中选择六个聊天机器人。

Fig. 3 图3
figure 3

The POE app interface with six chatbots accessible on the left side of the interface
POE 应用程序界面，界面左侧可访问六个聊天机器人

Full size image 全尺寸图像

The students were given 45 min to attempt a writing task using ChatGPT and other POE chatbots. The task was designed for students to demonstrate the range of writing skills and genre assessed in their EFL school curriculum, and to compel students to engage ChatGPT. (1) Students were instructed to write either a feature article or a letter to the editor. Figure 4 shows the prompts selected by the authors, taken from the 2023 Hong Kong university entrance examination for the EFL subject area (HKDSE), writing paper, which Hong Kong secondary school students take in their final year. (2) Students were instructed to write no more than 500 words on Google Docs, using their own words and words generated from POE chatbots. Students could prompt any POE chatbot in any way possible, as many times as necessary and use any chatbot output. (3) Students were instructed to differentiate their own words from AI words in their writing by highlighting words from each chatbot in a specific color. Figures 5 and 6 show a completed feature article and letter to the editor, respectively, following the color-coding scheme.
学生们有 45 分钟的时间尝试使用 ChatGPT 和其他 POE 聊天机器人完成写作任务。该任务旨在让学生展示其 EFL 学校课程中评估的写作技能和体裁范围，并迫使学生参与 ChatGPT。 (1) 学生被要求写一篇专题文章或给编辑的一封信。图4显示了作者选择的提示，摘自 2023 年香港大学入学考试 EFL 学科领域 (HKDSE) 写作试卷，这是香港中学生在最后一年参加的考试。 (2) 要求学生使用自己的单词和 POE 聊天机器人生成的单词在 Google Docs 上书写不超过 500 个单词。学生可以以任何可能的方式提示任何 POE 聊天机器人，根据需要多次提示并使用任何聊天机器人输出。 (3) 学生们被要求通过用特定颜色突出显示每个聊天机器人的单词来区分自己写作的单词和人工智能单词。图5和图6分别按照颜色编码方案显示了一篇完整的专题文章和给编辑的信。

Fig. 4 图4
figure 4

Two writing tasks, one for a feature article and the other for a letter to the editor task, each task comprising a topic heading, a picture and a written prompt
两项写作任务，一项是专题文章，另一项是给编辑的信任务，每个任务包括主题标题、图片和书面提示

Full size image 全尺寸图像
Fig. 5 图5
figure 5

A feature article written by a student with words from the Google-Palm chatbot in grey and the student’s words in black
学生撰写的专题文章，其中来自 Google-Palm 聊天机器人的文字为灰色，学生的文字为黑色

Full size image 全尺寸图像
Fig. 6 图6
figure 6

A letter to the editor written by a student and chatbots with the student’s words in black, words from the Sage chatbot in purple, ChatGPT in green, GPT-4 in blue, Claude + in red and Google-Palm in grey
学生和聊天机器人写给编辑的一封信，其中学生的文字为黑色，Sage 聊天机器人的文字为紫色，ChatGPT 为绿色，GPT-4 为蓝色，Claude + 为红色，Google-Palm 为灰色

Full size image 全尺寸图像

The research team had monitored student progress as students attempted the writing task during the workshop. Students were not required to complete the task during the workshop as the students and research team had agreed on a task completion deadline after the workshop.
当学生在研讨会期间尝试写作任务时，研究团队监控了学生的进度。学生不需要在研讨会期间完成任务，因为学生和研究团队已在研讨会后商定了任务完成期限。

2.3 Data collection
2.3数据收集

This mixed method study followed an embedded design (Creswell & Clark, 2007) where two sets of quantitative data and one set of qualitative data were collected in a workshop (see Table 3). In sum, the quantitative data from the pre-workshop questionnaire and the qualitative data from the think aloud protocols were collected to support quantitative data collected from the post-workshop questionnaire.
这项混合方法研究遵循嵌入式设计（Creswell & Clark， 2007 ），其中在研讨会上收集了两组定量数据和一组定性数据（参见表3 ）。总之，收集了来自研讨会前调查问卷的定量数据和来自大声思考协议的定性数据，以支持从研讨会后调查问卷中收集的定量数据。

Table 3 Data sources and purposes
表3 数据来源及目的
Full size table 全尺寸桌子

2.3.1 Pre-workshop questionnaire
2.3.1研讨会前调查问卷

To collect data on students’ learning motivation about ChatGPT before writing the task, a pre-workshop questionnaire, which also collected student background information, was developed. The learning motivation part comprised seven items with a six-point rating scheme (see Appendix), which was adapted from a measurement tool developed by Hwang and Chang (2011) to assess the motivation of fifth-grade primary school students towards a local culture course. The original scale has undergone thorough review, adoption, and adaptation by researchers (e.g., Cai et al., 2014; Huang et al., 2023) in diverse contexts to evaluate students' motivation. In accordance with their procedures, the authors modified the scale by replacing the course name with terms relevant to learning ChatGPT to ensure content validity. Furthermore, a pilot study involving 46 participants was conducted to establish the construct validity of this questionnaire. The results of a confirmatory factor analysis (CFA) yielded favorable indices: X2/df = 1.02, P(CMIN) = 0.422, CMIN/DF = 1.018, root mean square error of approximation (RMSEA) = 0.022, and comparative fit index (CFI) = 0.989, thereby confirming the construct validity. The reliability of the learning motivation questionnaire in this study was found to be 0.95, indicating a high level of internal consistency. As students are taught Chinese language and English language literacy in school, the questionnaire was delivered in English language and traditional Chinese language text. Students completed the questionnaire at the workshop, before the delivery of instructional materials. The questionnaire was introduced to students verbally, in English and in Cantonese Chinese, most students’ mother tongue. The research team monitored students while they completed the questionnaire and was available to answer any questions.
为了在编写任务之前收集有关学生关于 ChatGPT 的学习动机的数据，我们开发了一份研讨会前调查问卷，其中还收集了学生的背景信息。学习动机部分由七个项目组成，采用六分制评分方案（见附录），改编自Hwang和Chang（ 2011 ）开发的测量工具，用于评估五年级小学生学习当地文化课程的动机。 。原始量表经过了研究人员（例如，Cai et al., 2014 ；Huang et al., 2023 ）在不同背景下的彻底审查、采用和改编，以评估学生的动机。根据他们的程序，作者修改了量表，将课程名称替换为与学习 ChatGPT 相关的术语，以确保内容的有效性。此外，还进行了一项涉及 46 名参与者的试点研究，以确定该问卷的结构效度。验证性因子分析 (CFA) 的结果产生了有利的指数：X2/df = 1.02，P(CMIN) = 0.422，CMIN/DF = 1.018，近似均方根误差 (RMSEA) = 0.022，比较拟合指数 ( CFI) = 0.989，从而证实了结构的有效性。本研究学习动机问卷的信度为0.95，具有较高的内部一致性。由于学生在学校接受汉语和英语读写能力教育，因此问卷以英语和繁体中文文本进行。在交付教学材料之前，学生在研讨会上完成了调查问卷。调查问卷以英语和大多数学生的母语广东话口头形式向学生介绍。 研究团队在学生填写调查问卷时对其进行监控，并可以回答任何问题。

2.3.2 Think aloud protocols
2.3.2出声思考协议

To collect data on students’ cognitive load during the prompt engineering phase of writing the task, thinking aloud (TA), a research method where a student speaks their thoughts and feelings during an activity (Ericsson & Simon, 1993), was utilized. Scholars (Charters, 2003; Yoshida, 2008) have claimed that think aloud protocols provide insights into students’ cognitive load from demanding language tasks that can influence working memory and verbalization. In this way, students may not suffer great cognitive load if they can speak effortlessly and fluently.
为了在编写任务的即时工程阶段收集学生认知负荷的数据，使用了大声思考（TA），这是一种学生在活动期间说出他们的想法和感受的研究方法（Ericsson＆Simon， 1993 ）。学者们（Charters， 2003 ；Yoshida， 2008 ）声称，出声思考协议可以深入了解学生在高要求的语言任务中的认知负荷，这些任务会影响工作记忆和言语表达。这样，如果学生能够轻松、流利地说话，就可能不会承受很大的认知负担。

The authors randomly sampled nine students for the think-aloud method. Not least because of the smaller sample, this data was supplementary to retrospective data collection from a larger sample. Furthermore, the authors took a pragmatic view (Cotton & Gresty, 2006) to students’ think-aloud protocols, actively moderating them. At the workshop, before students attempted the task, the selected students were briefed on think-aloud protocols in English language and Cantonese Chinese language; and the authors demonstrated a protocol. The fourth author administered the think-aloud protocols, spending six minutes with each student, video-recording students’ iPad screens and iteratively asking students when they arrived at specific interaction points with a POE chatbot, (1) What do you think about this prompt? (visual cue: student has cursor in chatbot input box or is typing in chatbot input box) (2) What do you think about this output? (visual cue: chatbot has completed its output; and student is not typing anything) (3) How do you feel? (visual cue: student appears to have stopped answering question two) Students could answer in either or both English language and Cantonese Chinese language.
作者随机抽取了九名学生进行有声思考法。尤其是由于样本较小，该数据是对较大样本的回顾性数据收集的补充。此外，作者对学生的出声思考方案采取了务实的观点（Cotton & Gresty， 2006 ），并积极对其进行调节。在研讨会上，在学生尝试任务之前，向选定的学生简要介绍了英语和粤语的有声思维协议；作者展示了一个协议。第四位作者实施了有声思考协议，与每位学生共度六分钟，对学生的 iPad 屏幕进行视频录制，并反复询问学生何时到达与 POE 聊天机器人的特定交互点，(1) 您对此提示有何看法？ （视觉提示：学生将光标放在聊天机器人输入框中或正在聊天机器人输入框中键入）（2）您对此输出有何看法？ （视觉提示：聊天机器人已完成输出；学生没有输入任何内容） (3) 您感觉如何？ （视觉提示：学生似乎已停止回答第二个问题）学生可以用英语和粤语中的一种或两种语言回答。

Of the nine think-aloud protocols video-recorded on July 5 and 6, only the five collected on July 5 had sound. These five protocols were transcribed for each thought, the sequence of the thought, the timestamp on the video recording, and the chatbot and prompt used at the time.
在 7 月 5 日和 6 日录制的九个有声思考协议中，只有 7 月 5 日收集的五个有声音。这五个协议针对每个想法、想法的顺序、视频记录上的时间戳以及当时使用的聊天机器人和提示进行了转录。

2.3.3 Post-workshop questionnaire
2.3.3研讨会后调查问卷

A post-workshop questionnaire was the primary method to collect data on (1) learning motivation, (2) satisfaction, and (3) cognitive load. The seven learning motivation items were the same as those administered in the pre-workshop questionnaire except in the post-workshop items, the term “ChatGPT” had been replaced with the phrase “ChatGPT and other POE chatbots.” For instance, item 1 in the post-workshop questionnaire was, “I think learning ChatGPT and other POE chatbots is interesting and valuable [我認為學習ChatGPT和其他POE聊天機器人很有趣且有價值].” These terms were replaced because by the end of the task, students had been introduced to additional chatbots besides ChatGPT. The 14 satisfaction items were adapted from Fisher et al. (2010) (see Appendix). The eight, cognitive load items with a six-point Likert rating scheme were developed based on the measures of Paas (1992) and Sweller et al. (1998) (see Appendix). To ensure content validity, two experts with knowledge and expertise related to the construct of “satisfaction” were invited to evaluate the items for relevance, clarity, and comprehensiveness. Both experts confirmed the acceptability of the questionnaire, supporting its content validity. Additionally, the CFA on the pilot study showed that X2/df = 1.14, P(CMIN) = 0.21, CMIN/DF = 1.14, RMSEA = 0.07, and CFI = 0.97, confirming the construct validity. The Cronbach’s alpha value for this satisfaction questionnaire was 0.98, indicating a high level of consistency.
研讨会后调查问卷是收集以下数据的主要方法：(1) 学习动机、(2) 满意度和 (3) 认知负荷。七个学习动机项目与研讨会前调查问卷中的项目相同，只是在研讨会后项目中，术语“ChatGPT”已替换为短语“ChatGPT 和其他 POE 聊天机器人”。例如，研讨会后调查问卷中的第一项是，“我认为学习ChatGPT和其他POE聊天机器人很有趣且有价值[我认为学习ChatGPT和其他POE聊天机器人很有价值]。”这些术语被替换是因为在任务结束时，学生们已经了解了除 ChatGPT 之外的其他聊天机器人。 14 个满意度项目改编自 Fisher 等人。 （ 2010 ）（见附录）。八个认知负荷项目采用六点李克特评级方案，是根据 Paas ( 1992 ) 和 Sweller 等人的测量方法开发的。 （ 1998 ）（见附录）。为了确保内容的有效性，邀请了两位具有与“满意度”构建相关的知识和专业知识的专家来评估项目的相关性、清晰度和全面性。两位专家都确认了调查问卷的可接受性，支持其内容的有效性。此外，试点研究的 CFA 显示 X2/df = 1.14、P(CMIN) = 0.21、CMIN/DF = 1.14、RMSEA = 0.07 和 CFI = 0.97，证实了结构有效性。本次满意度调查的Cronbach's alpha值为0.98，一致性较高。

The cognitive load questionnaire consists of five items related to mental load and three items pertaining to mental effort. The original questionnaire, as presented in Hwang et al. (2013), was initially developed to assess the mental load and mental efforts of sixth-grade primary school students engaged in a game-based learning activity. Since then, this scale has been extensively examined, adopted, and adapted by researchers in various learning contexts to explore students' cognitive load (e.g., Dong et al., 2020; Hsu, 2017). Following their methodological guidelines, minor adjustments were made by substituting the term “learning activity” with “workshop” to measure students’ cognitive load, thereby ensuring content validity. Furthermore, the CFA conducted during the pilot study revealed that X2/df = 1.08, P(CMIN) = 0.37, CMIN/DF = 1.08, RMSEA = 0.058, and CFI = 0.99, thereby confirming the construct validity. The dimensions of mental load and mental effort exhibited high levels of internal consistency, with Cronbach’s alpha values of 0.97 and 0.95, respectively. The questionnaire was delivered in English language and traditional Chinese language text. Students completed the post-workshop questionnaire at the end of the workshop on Google Forms. Like the pre-workshop questionnaire, the post-workshop questionnaire was introduced to students verbally, in English and in Cantonese Chinese, the research team monitored students while they completed the questionnaire.
认知负荷问卷由与精神负荷相关的五个项目和与脑力努力相关的三个项目组成。原始调查问卷，如 Hwang 等人提出的。 （ 2013 ）最初是为了评估六年级小学生参与游戏学习活动的心理负荷和心理努力而开发的。从那时起，该量表被研究人员在各种学习环境中进行了广泛的研究、采用和调整，以探索学生的认知负荷（例如，Dong 等人， 2020 ；Hsu， 2017 ）。遵循他们的方法指导方针，进行了细微的调整，用“研讨会”代替“学习活动”一词来衡量学生的认知负荷，从而确保内容的有效性。此外，在试点研究期间进行的 CFA 显示 X2/df = 1.08、P(CMIN) = 0.37、CMIN/DF = 1.08、RMSEA = 0.058 和 CFI = 0.99，从而确认了结构有效性。精神负荷和精神努力维度表现出较高的内部一致性，Cronbach's alpha 值分别为0.97 和0.95。调查问卷以英文和繁体中文文本进行。学生在研讨会结束时通过 Google Forms 填写了研讨会后调查问卷。与工作坊前的问卷一样，工作坊后的问卷以口头方式向学生介绍，以英语和粤语进行，研究团队在学生填写问卷的同时对他们进行监控。

2.4 Data analysis
2.4数据分析

To investigate EFL students' learning motivation, cognitive load, and satisfaction after their active participation in the study, the authors analyzed post-workshop questionnaire data, employing basic descriptive statistics, including mean, standard deviation, minimum, and maximum values.
为了调查 EFL 学生积极参与研究后的学习动机、认知负荷和满意度，作者分析了研讨会后的问卷数据，采用基本的描述性统计数据，包括平均值、标准差、最小值和最大值。

To further investigate EFL students’ learning motivation, the descriptive statistics were applied to the pre-workshop questionnaire data. In addition, the Wilcoxon signed-rank test was employed to assess students’ motivation changes from pre-workshop to post-workshop, given that the data did not adhere to a normal distribution.
为了进一步调查英语学生的学习动机，对研讨会前的问卷数据进行了描述性统计。此外，鉴于数据不符合正态分布，采用 Wilcoxon 符号秩检验来评估学生从研讨会前到研讨会后的动机变化。

To further investigate EFL students’ cognitive load, the authors analyzed students’ think-aloud protocols, employing descriptive statistical measures such as the mean number of turns per student and the mean number of spoken words per turn, and standard deviation, minimum, and maximum values. This quantitative analysis was supplemented with representative quotes from students’ think aloud protocols. This analysis provides a fine-grained perspective to students’ cognitive load during the prompt engineering phase of writing with ChatGPT.
为了进一步调查 EFL 学生的认知负荷，作者分析了学生的有声思维方案，采用描述性统计方法，例如每个学生的平均轮数和每轮的平均口语单词数以及标准差、最小值和最大值价值观。这种定量分析还补充了学生大声思考协议中的代表性引文。该分析为学生在使用 ChatGPT 进行写作的即时工程阶段的认知负荷提供了细粒度的视角。

3 Results
3 个结果

3.1 Students’ learning motivation
3.1学生的学习动机

Twenty-one students had answered the post-workshop questionnaire and their median scores were found to be 35.00, within a range of 21 to 42. Twenty students had answered the pre-workshop questionnaire and their median scores for learning motivation among the students were observed to be 33.50, with a range spanning from 26 to 42. The results of the Wilcoxon signed-rank test, comparing the pre- and post-workshop learning motivation of the 20 EFL student cohort, are presented in Table 4. The result of the Wilcoxon signed-rank test (Z = 1.085, p = 0.278) indicates no significant difference in learning motivation between the pre- and post-workshop phases. However, it is noteworthy that the students exhibited an enhanced motivation to engage with ChatGPT and other POE chatbots in the post-workshop setting, as evidenced by a mean of 34.750 (SD = 6.604) in contrast to the mean of 33.850 (SD = 5.631) in the pre-workshop context.
21 名学生回答了工作坊后的调查问卷，他们的中位数分数为 35.00，范围在 21 至 42 之间。20 名学生回答了工作坊前的调查问卷，观察了他们的学习动机中位数分数为 33.50，范围从 26 到 42。Wilcoxon 符号秩检验的结果比较了 20 名 EFL 学生群体在研讨会之前和之后的学习动机，如表4所示。 Wilcoxon 符号秩检验的结果（Z = 1.085， p = 0.278）表明研讨会前后阶段的学习动机没有显着差异。然而，值得注意的是，学生们在研讨会结束后表现出与 ChatGPT 和其他 POE 聊天机器人互动的更强动机，平均值为 34.750 (SD = 6.604)，而平均值为 33.850 (SD = 5.631) ）在研讨会前的背景下。

Table 4 The result of the Wilcoxon Signed-rank test comparing the pre- and post-workshop motivation
表 4 比较研讨会前后动机的 Wilcoxon 符号秩检验结果
Full size table 全尺寸桌子

To gain deeper insights into the evolution of students' learning motivation, the authors undertook a visualization of their ratings across specific items, illustrated in Fig. 7. The analysis reveals an increase in learning motivation across five of the seven items. For instance, the average scores for items 1 and 2 exhibited a rise from 4.70 and 4.85 to 5.00, reflecting a heightened perception of the value and interest associated with learning about ChatGPT and other POE chatbots along with an augmented desire for further learning. Remarkably, although item 3 experienced a decrease in score, it remained at a conspicuously high level, with a rating of 5.00. In conclusion, the analysis of learning motivation partially corroborates that interactions with ChatGPT and other POE chatbots in the context of EFL writing have the potential to amplify students' motivation to advance their proficiency in utilizing chatbots.
为了更深入地了解学生学习动机的演变，作者对特定项目的评分进行了可视化，如图7所示。分析显示，七个项目中的五个项目的学习动机有所增加。例如，第 1 项和第 2 项的平均分从 4.70 和 4.85 上升到 5.00，反映出人们对学习 ChatGPT 和其他 POE 聊天机器人的价值和兴趣的认知增强，以及对进一步学习的渴望增强。值得注意的是，第3项虽然得分有所下降，但仍然处于明显的高水平，评分为5.00。总之，学习动机的分析部分证实了在 EFL 写作背景下与 ChatGPT 和其他 POE 聊天机器人的交互有可能增强学生的动机，以提高他们使用聊天机器人的熟练程度。

Fig. 7 图7
figure 7

Students’ learning motivation on each item with pre-workshop result on the left and post-workshop result on the right for each pair of bars
学生对每个项目的学习动机，每对条形图左侧为研讨会前结果，右侧为研讨会后结果

Full size image 全尺寸图像

3.2 Students’ cognitive load
3.2学生的认知负荷

Table 5 and Fig. 8 describe data from the post-workshop questionnaire about EFL students’ retrospective, self-reported cognitive load during the workshop. Intriguingly, students reported a relatively high level of cognitive load, with six out of the eight items returning an average rating of four out of six points. For instance, students attested to the challenging and effort-intensive nature of the workshop’s questions and tasks, a sentiment mirrored in their responses to items 2, 3, and 7. This cognitive load analysis suggests that students faced challenges when learning to write with ChatGPT and other POE chatbots to attempt a writing task.
表5和图8描述了研讨会后调查问卷中关于 EFL 学生在研讨会期间回顾性、自我报告的认知负荷的数据。有趣的是，学生们的认知负荷水平相对较高，八个项目中有六个项目的平均评分为四分（满分六分）。例如，学生证明了研讨会问题和任务的挑战性和费力性，这种情绪反映在他们对第 2、3 和 7 项的回答中。这种认知负荷分析表明，学生在学习使用 ChatGPT 写作时面临挑战和其他 POE 聊天机器人来尝试写作任务。

Table 5 The descriptive statistics regarding students’ cognitive load
表5 学生认知负荷的描述性统计
Full size table 全尺寸桌子
Fig. 8 图8
figure 8

Students’ cognitive load scores on each item
学生在每个项目上的认知负荷分数

Full size image 全尺寸图像

Analysis of five students’ think aloud protocols during the prompt engineering phase of their writing with ChatGPT provides some corroborative evidence for students’ high cognitive load. In general, students were not speaking effortlessly and freely. Figure 9 illustrates the number of spoken turns that each student took during a six-minute timespan. Students took on average 13 turns, with a range from five turns (n = 1) to 17 turns (n = 2). Figure 9 also illustrates the average number of words that each student spoke per turn. While Students 1A, 2A and 3A wrote ChatGPT prompts exclusively in English language and delivered think aloud protocols exclusively in English language, Students 3A and 3B wrote ChatGPT prompts exclusively in Chinese language and delivered think aloud protocols almost exclusively in Chinese language. Therefore, the authors translated these students’ words into the English language before preparing descriptive statistics. Students spoke on average 11 words per turn, with a range from five words per turn (n = 1) to 25 words per turn (n = 1).
对五名学生在使用 ChatGPT 进行写作的即时工程阶段的出声思考方案进行的分析为学生的高认知负荷提供了一些确凿的证据。总体而言，学生们的发言并不轻松、自由。图9显示了每个学生在六分钟时间内的发言次数。学生平均轮了 13 轮，范围从 5 轮 ( n = 1) 到 17 轮 ( n = 2)。图9还说明了每个学生每轮所说的平均单词数。学生 1A、2A 和 3A 仅用英语编写 ChatGPT 提示并仅用英语交付出声思考方案，而学生 3A 和 3B 仅用中文编写 ChatGPT 提示并几乎仅用中文交付出声思考方案。因此，作者在准备描述性统计之前，将这些学生的话翻译成英语。学生平均每轮说 11 个单词，范围从每轮 5 个单词 ( n = 1) 到每轮 25 个单词 ( n = 1)。

Fig. 9 图9
figure 9

A stacked bar chart for each student with each bar on the bottom showing the number of turns a student had taken and on the top showing the mean number of words spoken per turn
每个学生的堆叠条形图，底部的每个条形显示学生已进行的轮数，顶部的每个条形显示每轮所说的平均单词数

Full size image 全尺寸图像

Table 6 shows representative turns for each student. Each student’s turns represent a complete instance of a prompt engineering phase, showing their answers to the questions (1) what do you think about this prompt? (2) what do you think about this output? and (3) how do you feel? Each student’s turns were also selected as the most representative of the average number of words that the student spoke per turn.
表6显示了每个学生的代表性回合。每个学生轮流代表提示工程阶段的完整实例，显示他们对问题的答案（1）您对此提示有何看法？ （2）你对这个输出有何看法？ (3) 你感觉如何？每个学生的轮次也被选为最能代表该学生每轮所说的平均单词数的轮次。

Table 6 Representative turns
表6 代表性匝数
Full size table 全尺寸桌子

3.3 Students’ satisfaction
3.3学生满意度

Table 7 and Fig. 10 offer insights into students' satisfaction concerning the workshop where they learned to write with ChatGPT. On the whole, students expressed a high level of satisfaction throughout the workshop, as all survey items garnered an average rating surpassing 5.40 on a 7.0-point scale. As delineated in Fig. 10, students conveyed a robust sense of engagement and enjoyment in the workshop, as evidenced by their responses to items 8, 10, and 12. Furthermore, they reported a noteworthy level of focus, enthusiasm, and confidence in their ability to assimilate, retain, and apply the workshop content, exemplified by their responses to items 1, 2, 3, 4, 6, 7, and 11.
表7和图10深入了解了学生对学习使用 ChatGPT 进行写作的研讨会的满意度。总体而言，学生们对整个研讨会表示了很高的满意度，所有调查项目的平均评分都超过了 5.40（满分为 7.0）。如图10所示，学生们在研讨会上表达了强烈的参与感和享受感，这从他们对第 8、10 和 12 项的回答中可以看出。此外，他们对自己的工作表现出高度的专注、热情和信心。吸收、保留和应用研讨会内容的能力，例如他们对第 1、2、3、4、6、7 和 11 项的回答。

Table 7 The descriptive statistics regarding students’ satisfaction
表7 学生满意度的描述性统计
Full size table 全尺寸桌子
Fig. 10 图10
figure 10

Students’ satisfaction scores on each item
学生对每个项目的满意度得分

Full size image 全尺寸图像

4 Discussion
4讨论

This study has explored EFL students’ perceptions about learning to write with ChatGPT in terms of students’ motivation to learn, cognitive load, and satisfaction. The specific sample and context are Hong Kong EFL secondary students in a workshop where they were introduced to ChatGPT and prompt engineering, and attempted a 500-word writing task using ChatGPT for support. The results from the pre- and post-workshop questionnaires and from think aloud protocols provide insights into how using ChatGPT in writing impacts EFL students’ motivation to learn about ChatGPT, EFL students’ cognitive load when writing with ChatGPT and students’ satisfaction with the experience of writing with ChatGPT. The following are the major findings.
本研究从学习动机、认知负荷和满意度方面探讨了 EFL 学生对使用 ChatGPT 学习写作的看法。具体的样本和背景是香港 EFL 中学生在一个研讨会上，他们在研讨会上了解了 ChatGPT 和提示工程，并尝试使用 ChatGPT 进行 500 字的写作任务作为支持。研讨会前后的问卷调查以及出声思考协议的结果提供了关于在写作中使用 ChatGPT 如何影响 EFL 学生学习 ChatGPT 的动机、EFL 学生使用 ChatGPT 写作时的认知负荷以及学生对体验的满意度的见解。使用 ChatGPT 进行写作。以下是主要发现。

4.1 Major findings
4.1主要发现

The Wilcoxon signed-rank test results revealed no significant difference in students' motivation to learn about ChatGPT from pre- to post-workshop. However, a slight increase in the mean scores for post-workshop motivation suggests that students may have had a more favorable attitude towards learning about ChatGPT after engaging in the workshop activities. The widespread appeal of ChatGPT sparked considerable interest among students, motivating their voluntary participation in the workshop and explaining their initially high levels of motivation, as in line with the findings of Chan and Hu (2023), who reported a willingness among most students to utilize ChatGPT. This highlights the potential of ChatGPT's novelty and interactive nature to stimulate motivation for EFL writing. The integration of ChatGPT into EFL writing bridges the gap between cutting-edge technology and students' academic learning, establishing relevance and potentially contributing to increased motivation. This integration is also consistent with prior research suggesting that the incorporation of innovative technologies can enhance students' motivation to engage with learning materials (Kim & Lee, 2023; Shim et al., 2023). Furthermore, ChatGPT's features, such as generating helpful content tailored to human needs and facilitating interactive conversations, provide students with a high level of satisfaction, as evidenced by their responses to satisfaction questionnaires. This satisfaction may further explain their enhanced motivation to learn. However, it is important to acknowledge the possibility of a "novelty effect" contributing to the slight increase in motivation, which refers to the heightened motivation or perceived usability of a technology due to its novelty or newness (Koch et al., 2018). To substantiate this hypothesis, a longitudinal study is necessary to explore how students' learning motivation may evolve as they become accustomed to ChatGPT.
Wilcoxon 符号排名测试结果显示，在研讨会之前和之后，学生学习 ChatGPT 的动机没有显着差异。然而，研讨会后动机的平均得分略有增加，表明学生在参加研讨会活动后可能对学习 ChatGPT 抱有更有利的态度。 ChatGPT 的广泛吸引力激发了学生的极大兴趣，激发了他们自愿参加研讨会，并解释了他们最初的高动机，这与 Chan 和 Hu（ 2023 ）的研究结果一致，他们报告称大多数学生愿意使用聊天GPT。这凸显了 ChatGPT 的新颖性和互动性在激发 EFL 写作动机方面的潜力。 ChatGPT 与 EFL 写作的整合弥合了尖端技术和学生学术学习之间的差距，建立了相关性并可能有助于提高动机。这种整合也与之前的研究一致，即创新技术的结合可以增强学生学习材料的积极性（Kim & Lee， 2023 ；Shim 等人， 2023 ）。此外，ChatGPT 的功能（例如生成适合人类需求的有用内容和促进互动对话）为学生提供了很高的满意度，正如他们对满意度调查问卷的回答所证明的那样。这种满足感可能进一步解释了他们学习动机的增强。 然而，重要的是要承认“新颖效应”可能会导致动机略有增加，这是指由于技术的新颖性或新颖性而增强的动机或感知的可用性（Koch 等人， 2018 ）。为了证实这一假设，有必要进行一项纵向研究来探索学生的学习动机在习惯 ChatGPT 后如何演变。

Notably, this study has found students experienced heavy cognitive load when writing with ChatGPT. Specifically, the think-aloud protocol evidence suggests that students’ experience heavy cognitive load during the prompt engineering phase of writing with ChatGPT. Additionally, the think-aloud protocol evidence suggests the heavy cognitive load stems from neither the demands of writing in English, as students could write ChatGPT prompts in Chinese language, nor from the medium of instruction, as verbal and written instructions were delivered in English language and Chinese language, nor from the think-aloud protocols as students could perform protocols in either English or Chinese language. On the other hand, it is possible the heavy cognitive load stems from the basic cognitive processes associated with writing (Flower & Hayes, 1981) such as planning, drafting and reviewing. Another possibility is that the workshop’s time constraint influenced students heavy cognitive load as Shim et al. (2023) suggested that novices unfamiliar with chatbots need more time to follow instruction and to keep pace in a workshop format. Alternatively, think-aloud protocols have been criticized as providing an artificial and incomplete view of cognitive activity during writing, although they can provide insights into writing and writing response practices (Hyland, 2019).
值得注意的是，这项研究发现学生在使用 ChatGPT 写作时经历了沉重的认知负担。具体来说，有声思考协议证据表明，学生在使用 ChatGPT 进行写作的即时工程阶段会经历沉重的认知负担。此外，有声思考方案的证据表明，沉重的认知负荷既不是源于英语写作的要求（因为学生可以用中文写出 ChatGPT 提示），也不是源于教学媒介（因为口头和书面指令都是用英语进行的）和中文，也不是出声思考协议，因为学生可以用英语或中文执行协议。另一方面，沉重的认知负荷可能源于与写作相关的基本认知过程（Flower & Hayes， 1981 ），例如计划、起草和审阅。另一种可能性是，研讨会的时间限制影响了学生沉重的认知负担，正如 Shim 等人一样。 ( 2023 ) 建议不熟悉聊天机器人的新手需要更多时间来遵循说明并跟上研讨会形式的步伐。另外，有声思考协议也被批评为在写作过程中提供了一种人为的、不完整的认知活动视图，尽管它们可以提供对写作和写作反应实践的见解（Hyland， 2019 ）。

Importantly, the finding highlights the possibly high cognitive demands of integrating ChatGPT into the EFL writing classroom. Cognitive load theory posits that for effective learning to occur, instructional design should manage cognitive load to prevent overloading the learner's working memory (Sweller et al., 1998). Therefore, this study supports existing recommendations to optimize instruction for AI in the classroom so that students engage level-appropriate material and tasks yet are challenged to advance their cognitive boundaries (Walter, 2024). In the context of students writing with ChatGPT in the EFL classroom, teachers can lead students from comprehension-based tasks to controlled production to more communicative tasks (Nunan, 1989). Teachers can also provide materials on how to craft prompts and schematas by which students can evaluate output, because extensive reading must support EFL writing skills (Hyland, 2019). In addition, teachers may provide students with more time to engage in materials and tasks. By intentionally reducing EFL students’ cognitive load, teachers better position their students to benefit from ChatGPT in the writing classroom.
重要的是，这一发现凸显了将 ChatGPT 整合到 EFL 写作课堂中可能存在的高认知要求。认知负荷理论认为，为了实现有效的学习，教学设计应该管理认知负荷，以防止学习者的工作记忆超载（Sweller 等， 1998 ）。因此，这项研究支持现有的优化人工智能课堂教学的建议，以便学生参与适合水平的材料和任务，同时面临着提升他们的认知界限的挑战（Walter， 2024 ）。在学生在 EFL 课堂​​上使用 ChatGPT 写作的背景下，教师可以引导学生从基于理解的任务转向受控生产，再转向更具交流性的任务（Nunan， 1989 ）。教师还可以提供有关如何制作提示和图式的材料，以便学生评估输出，因为广泛的阅读必须支持 EFL 写作技能（Hyland， 2019 ）。此外，教师可以为学生提供更多的时间来完成材料和任务。通过有意减少 EFL 学生的认知负荷，教师可以更好地让学生在写作课堂上从 ChatGPT 中受益。

Students expressed high satisfaction with the workshop overall. The analysis of satisfaction partially supports that engagement with ChatGPT and other POE chatbots in the context of an EFL writing classroom fosters a highly gratifying and enriching educational experience for students. It corroborates prior ChatGPT research where students reported high satisfaction with guided ChatGPT interactions (Amaro et al., 2023) and with writing feedback from ChatGPT (Escalante et al., 2023). The distinctive characteristics of ChatGPT may contribute to the observed high level of satisfaction, as previously mentioned. Firstly, ChatGPT is trained using the reinforcement learning from human feedback method (RLHF; Stiennon et al., 2020). This training approach modifies ChatGPT's language modeling objective, shifting its focus from predicting the next token on a webpage to providing helpful and safe responses based on user instructions (Ouyang et al., 2022, p. 2). Consequently, ChatGPT excels at generating responses that are aligned with human preferences and priorities. In the context of this study on EFL writing, ChatGPT effectively produces responses that cater to students' preferences and facilitate their writing process, thereby explaining their heightened satisfaction. Secondly, ChatGPT exhibits a conversational nature. Previous studies have shown that extended interactions with conversational agents can enhance users' overall experience (Jacq et al., 2016). Similarly, the students' frequent interaction with the human-like ChatGPT in this study can be considered a contributing factor to their elevated satisfaction. However, although high satisfaction is an important predictor of future engagement and positive learning outcomes, it alone does not guarantee learning.

4.2 Implications, limitations and future research

This study has allowed educators to better understand the integration of ChatGPT into the English as a foreign language (EFL) writing classroom in terms of students' experiences and perceptions. Specifically, educators can better understand the potential of ChatGPT to enhance the EFL writing classroom by motivating and engaging students. Furthermore, a workshop format can be a suitable way to integrate ChatGPT into the EFL writing classroom. However, educators should carefully scaffold instruction, especially in teaching prompt engineering skills, so as to manage students’ cognitive load. Educators can consider an iterative design process of activities and instructional materials. Careful design ensures that ChatGPT use supports writing without overwhelming students. Educators may begin by identifying the target text type for students to write, adopting an explicit approach to writing that text type, mapping ChatGPT capabilities to that writing approach and developing genre writing and prompt engineering instructional materials, such as worksheets, and activities such as educators and students jointly constructing prompts. Implementing extensively scaffolded instruction to better integrate ChatGPT in the writing classroom will require more contact time.

Although this exploratory study provides a meaningful window into EFL students' perceptions about learning to write with ChatGPT, its limitations should be considered when interpreting the results. The sample size was relatively small, in terms of number of students, schools and instructional time, which may limit the generalizability of the findings. The sample was all female. Further research could involve larger, and more diverse samples, including males, to further validate the findings. Furthermore, research could explore how students' perceptions evolve with prolonged engagement. For instance, longitudinal studies could examine how students' ability to manage cognitive load improves with prolonged exposure to ChatGPT. Likewise, further research could explore differences in students’ perceptions between two types of workshops. For instance, student perceptions from this study’s workshop can be compared to those from a workshop with adjusted instruction aimed to reduce cognitive load, such as improved prompt engineering instruction. Another limitation is the reliance on self-report measures, which may not fully capture students' perceptions. Future research could incorporate additional measures of student perceptions, such as additional observational data besides screen recordings to further validate self-report measures.

4.3 Conclusion

This mixed-methods study explored Hong Kong secondary school EFL students' experiences and perceptions of learning to write a composition with ChatGPT's support in a workshop format. Key findings revealed that, although not statistically significant, students' mean motivation scores to learn about ChatGPT increased slightly from pre- to post-workshop, suggesting ChatGPT's potential to engage students. However, students reported high cognitive load in the workshop, notably when writing with a machine-in-the-loop. This highlights the need for educators to carefully scaffold instruction and activities to reduce students' cognitive load in the classroom. Nonetheless, students expressed high overall satisfaction with the workshop experience.

This study adds to the empirical research on EFL student experiences and perceptions of writing with ChatGPT. The findings provide insights for educators on the motivational benefits and cognitive demands of integrating ChatGPT in the writing classroom. Furthermore, the study proposes developing EFL students' prompt engineering skills alongside genre writing skills to optimize ChatGPT's support at different stages of learning to write a text type. Overall, the study advances how to enhance EFL classroom writing with ChatGPT integration. Future studies could involve larger, more diverse samples, explore longitudinal effects, and compare varied instructional designs for integrating ChatGPT in EFL writing.