这是用户在 2024-5-26 13:33 为 https://app.immersivetranslate.com/pdf-pro/82965e37-7e4c-4106-a56c-d88fed97021f 保存的双语快照页面，由 沉浸式翻译 提供双语支持。了解如何保存？
2024_05_26_76c5c98ffbb4ce6579adg

Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020
在线高等教育中的人工智能：2011 年至 2020 年实证研究系统回顾

Fan Ouyang Luyi Zheng Pengcheng Jiao
Fan Ouyang Luyi Zheng Pengcheng Jiao

Received: 5 October 2021 / Accepted: 27 January 2022 / Published online: 26 February 2022
收到：2021 年 10 月 5 日 / 接受：2022 年 1 月 27 日 / 在线发表：2022 年 2 月 26 日收到：2021 年 10 月 5 日 / 接受：2022 年 1 月 27 日 / 在线发表：2022 年 2 月 26 日
O The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022
O 作者，独家授权 Springer Science+Business Media, LLC（Springer Nature 2022 的一部分

Abstract 摘要

As online learning has been widely adopted in higher education in recent years, artificial intelligence (AI) has brought new ways for improving instruction and learning in online higher education. However, there is a lack of literature reviews that focuses on the functions, effects, and implications of applying in the online higher education context. In addition, what AI algorithms are commonly used and how they influence online higher education remain unclear. To fill these gaps, this systematic review provides an overview of empirical research on the applications of in online higher education. Specifically, this literature review examines the functions of in empirical researches, the algorithms used in empirical researches and the effects and implications generated by empirical research. According to the screening criteria, out of the 434 initially identified articles for the period between 2011 and 2020, 32 articles are included for the final synthesis. Results find that: (1) the functions of AI applications in online higher education include prediction of learning status, performance or satisfaction, resource recommendation, automatic assessment, and improvement of learning experience; (2) traditional AI technologies are commonly adopted while more advanced techniques (e.g., genetic algorithm, deep learning) are rarely used yet; and (3) effects generated by AI applications include a high quality of AI-enabled prediction with multiple input variables, a high quality of AI-enabled recommendations based on student characteristics, an improvement of students' academic performance, and an improvement of online engagement and participation. This systematic review proposes the following theoretical, technological, and practical implications: (1) the integration of educational and learning theories into AI-enabled online learning; (2) the adoption of advanced AI technologies to collect and analyze real-time process data; and (3) the implementation of more empirical research to test actual effects of AI applications in online higher education.
近年来，随着在线学习在高等教育中的广泛应用，人工智能（AI）为改进在线高等教育的教学和学习带来了新的方法。然而，目前还缺乏重点研究 在在线高等教育中应用的功能、效果和影响的文献综述。此外，常用的人工智能算法有哪些，它们如何影响在线高等教育，这些问题仍不清楚。为了填补这些空白，本系统综述概述了 在在线高等教育中应用的实证研究。具体而言，本文献综述探讨了 在实证研究中的功能、实证研究中使用的算法以及实证研究产生的效果和影响。根据筛选标准，在 2011 年至 2020 年期间初步确定的 434 篇文章中，有 32 篇文章被纳入最终综述。结果发现(1) 人工智能应用在在线高等教育中的功能包括预测学习状态、成绩或满意度，推荐资源，自动评估，以及改善学习体验；(2) 传统的人工智能技术被普遍采用，而更先进的技术（如遗传算法、深度学习等）还很少被使用；(3) 人工智能应用产生的效果包括在多个输入变量下的高质量人工智能预测，基于学生特征的高质量人工智能推荐，学生学业成绩的提高，以及在线参与度和参与度的提高。本系统综述提出了以下理论、技术和实践意义：（1）将教育和学习理论融入人工智能在线学习；（2）采用先进的人工智能技术收集和分析实时过程数据；（3）开展更多实证研究，检验人工智能应用在在线高等教育中的实际效果。

Keywords Artificial Intelligence in Education Systematic review Online higher education Online learning Empirical research
关键词 人工智能教育 系统综述 在线高等教育 在线学习 实证研究

1 Introduction 1 引言

Advances in the Internet, wireless communication, and computing technologies have shed light on educational changes in online higher education, particularly the application of Artificial Intelligence in Education (AIEd) in recent years (Chen et al., 2020a, b; Ouyang & Jiao, 2021). Online and distance learning refers to delivering lectures, virtual classroom meetings, and other teaching materials and activities via the Internet (Harasim, 2000; Holmberg, 2005). This educational model has been extensively integrated into higher education to transform instruction and learning modes as well as provide fair educational opportunities to online learners (Hu, 2021; Liu et al., 2020; Mubarak et al., 2020; Yang et al., 2014). In the online education context, AI applications (e.g. intelligent tutoring systems, teaching robots, learning analytics dashboards, adaptive learning systems) have been used to promote online students' learning experience, performance, and quality (Chen et al., 2020a, b; Hinojo-Lucena et al., 2019). Varied AIEd techniques (e.g., natural language processing, artificial neural networks, machine learning, deep learning, genetic algorithm) have been implemented to create intelligent learning environments for behavior detection, prediction model building, learning recommendation, etc. (Chen et al., 2020a, b; Rowe, 2019). Overall, the applications of AI systems and technologies have transformed online higher education and have provided opportunities and challenges for improving higher education quality. Previous reviews have provided substantial insight into the AIEd field. For instance, existing review work has summarized the trends of AIEd (Tang et al., 2021; Xie et al., 2019; Zhai et al., 2021), applications (Alyahyan & Düşsegör, 2020; Hooshyar et al., 2016; Liz-Domínguez et al., 2019; Shahiri et al., 2015), theoretical paradigms (Ouyang & Jiao, 2021) and AI roles in education ( Xu & Ouyang, 2021). However, there is limited literature reviews that examine the purposes and effects of applying AI techniques in the online higher education context. More important, a major challenge is to gain a deep understanding of the empirical effect of AI applications in online higher education. To achieve this purpose, this systematic review collects, reviews, and summarizes the empirical research of in online higher education with particular aims to analyze the application purposes, the AI algorithms used, and effects of AI techniques generated in online higher education.
互联网、无线通信和计算技术的发展为在线高等教育的教育变革带来了曙光，特别是近年来人工智能在教育领域的应用（Chen et al.在线和远程学习是指通过互联网提供讲座、虚拟课堂会议以及其他教学材料和活动（Harasim，2000；Holmberg，2005）。这种教育模式已广泛融入高等教育，以改变教学和学习模式，并为在线学习者提供公平的教育机会（Hu，2021；Liu 等人，2020；Mubarak 等人，2020；Yang 等人，2014）。在在线教育方面，人工智能应用（如智能辅导系统、教学机器人、学习分析仪表板、自适应学习系统）已被用于提升在线学生的学习体验、成绩和质量（Chen 等，2020a，b；Hinojo-Lucena 等，2019）。各种人工智能教育技术（如自然语言处理、人工神经网络、机器学习、深度学习、遗传算法）已被用于创建智能学习环境，以进行行为检测、预测模型构建、学习推荐等（Chen 等，2020a，b；Rowe，2019）。总体而言，人工智能系统和技术的应用已经改变了在线高等教育，并为提高高等教育质量提供了机遇和挑战。以往的综述为人工智能教育领域提供了大量见解。例如，现有的综述工作总结了 AIEd 的发展趋势（Tang 等，2021；Xie 等，2019；Zhai 等，2021）、应用（Alyahyan & Düşsegör，2020；Hooshyar 等，2016；Liz-Domínguez 等，2019；Shahiri 等，2015）、理论范式（Ouyang & Jiao，2021）和人工智能在教育中的作用（Xu & Ouyang，2021）。然而，研究在线高等教育中应用人工智能技术的目的和效果的文献综述十分有限。更重要的是，如何深入了解在线高等教育中人工智能应用的实证效果是一大挑战。为此，本系统性综述收集、回顾和总结了 在在线高等教育中的实证研究，尤其旨在分析人工智能技术在在线高等教育中的应用目的、使用的人工智能算法以及产生的效果。

2 Literature review 2 文献综述

AIEd refers to the use of AI technologies or applications in educational settings to facilitate instruction, learning, and decision making processes of stakeholders, such as students, instructors, and administrators (Hwang et al., 2020). In online higher education, AI can support instructional design and development by providing automatic learning resources or paths (Christudas et al., 2018), offering automatic assessments (Aluthman, 2016) or predictions of student performance
人工智能教育是指在教育环境中使用人工智能技术或应用，以促进学生、教师和管理人员等利益相关者的教学、学习和决策过程（Hwang et al.）在在线高等教育中，人工智能可以通过提供自动学习资源或路径（Christudas et al.
(Almeda et al., 2018; Moreno-Marcos et al., 2019). From the instructional perspective, AI can play the role as a tutor to observe students' learning processes, analyze their learning performances, and provide chances for instructors to get rid of repetitive and tedious teaching tasks (Chen et al., 2020a, b; Hwang et al., 2020). Moreover, from the learner perspective, One of the crucial objectives of AIEd is to providing personalized learning guidance or support based on students' learning status, preferences, or personal characteristics (Hwang et al., 2020). For instance, AIEd can provide learning materials or paths based on students' needs (Christudas et al., 2018), diagnose students' strengths, weaknesses, or knowledge gaps (Liu et al., 2017), or provide automated feedback and promoting collaboration between students (Aluthman, 2016; Benhamdi et al., 2017; Zawacki-Richter et al., 2019). Furthermore, AIEd can help educational administrators make decisions about course development, pedagogical design and academic transformation. For example, AI algorithm models can mine and analyze available educational data from higher education system database to understand course status, student learning performance, which can help administrators or decision-makers to make changes needed in the course (George & Lal, 2019). In summary, AI-enhanced technology has played an essential role in education from the instructor, learner and administrator perspectives, with its potential to open new opportunities and challenges for higher education transformation.
(Almeda 等人，2018；Moreno-Marcos 等人，2019）。从教学角度看，人工智能可以扮演辅导员的角色，观察学生的学习过程，分析他们的学习表现，为教师提供摆脱重复乏味的教学任务的机会（Chen 等，2020a，b；Hwang 等，2020）。此外，从学习者的角度来看，AIEd 的重要目标之一是根据学生的学习状况、偏好或个人特征提供个性化的学习指导或支持（Hwang 等人，2020）。例如，AIEd 可以根据学生的需求提供学习材料或路径（Christudas 等人，2018 年），诊断学生的优势、劣势或知识差距（Liu 等人，2017 年），或提供自动反馈并促进学生之间的合作（Aluthman，2016 年；Benhamdi 等人，2017 年；Zawacki-Richter 等人，2019 年）。此外，AIEd 还能帮助教育管理者做出有关课程开发、教学设计和学术转型的决策。例如，人工智能算法模型可以从高等教育系统数据库中挖掘和分析可用的教育数据，了解课程状况、学生学习成绩，从而帮助管理者或决策者对课程进行必要的修改（George & Lal，2019）。总之，从教师、学习者和管理者的角度来看，人工智能增强技术在教育中发挥了至关重要的作用，其潜力为高等教育转型带来了新的机遇和挑战。
Multiple AI algorithms have been applied in higher education to facilitate automatic recommendation, academic prediction, or assessment. For example, Sequential Pattern Mining (SPM) has been utilized in recommender systems for capturing historical learning sequence patterns in learner interactions with the system and discovering suitable recommendation items for learners' learning sequences (Romero et al., 2013a). Evolutionary algorithms such as Genetic Algorithms (GA), Particle Swarm Optimization (PSO) and Ant Colony Optimization (ACO) have been used for learning content optimization (Christudas et al., 2018). Machine Learning (ML) has been used in academic prediction, such as predicting the academic success of students in online courses, whether students would successfully complete their college degree, or predict students' selection of courses in higher education (Rico-Juan et al. 2019). Lykourentzou et al. (2009) used three machine learning techniques, namely feed-forward neural networks, support vector machines and probabilistic ensemble simplified fuzzy ARTMAP to predict dropout-prone students in early stages of the e-learning course. Moseley and Mead (2008) used a machine learning technique called decision trees to predict student attrition in higher educational nursing institutions. Natural language processing (NLP) has been used for code detection or emotional analysis. For example, Rico-Juan et al. (2019) adopted NLP for automatic detection of inconsistencies between numerical scores and textual feedback in peer-assessment process. In summary, different AI algorithms have been used in AIEd to achieve automatic recommendation, academic prediction, or assessment functions in order to improve instruction and learning quality.
多种人工智能算法已被应用于高等教育领域，以促进自动推荐、学业预测或评估。例如，序列模式挖掘（SPM）已被应用于推荐系统，用于捕捉学习者与系统交互过程中的历史学习序列模式，并为学习者的学习序列发现合适的推荐项目（Romero 等人，2013a）。遗传算法（GA）、粒子群优化（PSO）和蚁群优化（ACO）等进化算法已被用于学习内容优化（Christudas 等人，2018 年）。机器学习（ML）已被用于学术预测，如预测学生在在线课程中的学业成功率、学生是否能顺利完成大学学业，或预测学生在高等教育中的选课情况（Rico-Juan 等，2019 年）。Lykourentzou 等人（2009 年）使用了三种机器学习技术，即前馈神经网络、支持向量机和概率集合简化模糊 ARTMAP，来预测电子学习课程早期阶段容易辍学的学生。Moseley 和 Mead（2008 年）使用一种名为决策树的机器学习技术来预测高等教育护理机构的学生流失率。自然语言处理（NLP）已被用于代码检测或情感分析。例如，Rico-Juan 等人（2019 年）采用 NLP 自动检测互评过程中数字分数与文本反馈之间的不一致性。总之，不同的人工智能算法已被用于人工智能教育，以实现自动推荐、学业预测或评估功能，从而提高教学和学习质量。
Although there are existing systematic reviews on AIEd (e.g., AIEd trends, paradigms, tools, or applications) (Ouyang & Jiao, 2021; Tang et al., 2021), there is an inadequacy of review work examining AIEd in the higher education context. Among a collection of 37 AIEd review articles published between 2011 and
尽管目前已有一些关于人工智能教育（如人工智能教育的趋势、范式、工具或应用）的系统性综述（欧阳和焦，2021；唐等人，2021），但对高等教育背景下的人工智能教育进行研究的综述还不够充分。在2011年至2011年期间发表的37篇人工智能教育评论文章中
Fig. 1 The existing literature review of AIEd
图 1 现有的 AIEd 文献综述
2021, only 6 review articles focus on higher education, published in the years of 2019 and 2020 (see Fig. 1). Among those six review articles, Hinojo-Lucena et al. (2019) used the bibliometric method to review the applications of AI in higher education. This review analyzed the number of authors, main source titles, organizations, authors, and countries of AIEd in higher education. Zawacki-Richter et al. (2019) synthesized 146 articles about the application of AI in higher education and concluded four major areas, namely profiling and prediction, intelligent tutoring systems, assessment, and evaluation, as well as adaptive learning systems. However, this review work did not conduct further analysis for examining the effects of AI in online higher education. Moreno-Marcos et al. (2018) used a systematic literature review to examine the AI models used for performance prediction in MOOCs. This review found that AI algorithms used for prediction were: regression, support vector machines (SVM), decision trees (DTs), random forest (RF), naive Bayes (NB), gradient boosting machine (GBM), neural networks (NN), etc. We concluded that existing literature review work mainly focuses on the application of AIEd in general, and few work specifically focuses on the online higher education. Among those work that focus on AI in online higher education, they mainly focus on describing the applications of AI in a specific educational context (e.g., MOOCs), which resulted in the lack of a holistic picture of the AIEd trends, categorizations, and applications in online higher education.
2021年，只有6篇评论文章关注高等教育，发表于2019年和2020年（见图1）。在这6篇综述文章中，Hinojo-Lucena等人（2019）采用文献计量学方法对人工智能在高等教育中的应用进行了综述。这篇综述分析了 AIEd 在高等教育中的作者数量、主要来源标题、组织、作者和国家。Zawacki-Richter等人（2019）综合了146篇有关人工智能在高等教育中应用的文章，总结出四大领域，即剖析与预测、智能辅导系统、评估与评价以及自适应学习系统。不过，这篇综述并未对人工智能在在线高等教育中的应用效果进行进一步分析。Moreno-Marcos 等人（2018）通过系统的文献综述研究了用于 MOOCs 成绩预测的人工智能模型。该综述发现，用于预测的人工智能算法有：回归、支持向量机（SVM）、决策树（DT）、随机森林（RF）、天真贝叶斯（NB）、梯度提升机（GBM）、神经网络（NN）等。我们的结论是，现有的文献综述工作主要集中在一般的人工智能教育应用上，很少有专门针对在线高等教育的工作。在这些关注人工智能在在线高等教育中应用的文献中，它们主要集中于描述人工智能在特定教育环境（如MOOCs）中的应用，这导致对人工智能教育在在线高等教育中的趋势、分类和应用缺乏全面的了解。
As an effort to further understand in online higher education, this systematic review examines empirical research of AI applications in online higher education from the instructional and learning perspective and investigates the functions of AI applications, algorithms used, and the effects of AI on the instruction and learning process within online higher education. To be specific, this review focuses on the following three research questions:
为了进一步了解 在在线高等教育中的应用，本系统综述从教学和学习的角度出发，考察了在线高等教育中人工智能应用的实证研究，研究了人工智能应用的功能、使用的算法以及人工智能对在线高等教育教学和学习过程的影响。具体而言，本综述主要关注以下三个研究问题：
RQ1: What are the functions of AI applications in online higher education?
问题 1：在线高等教育中的人工智能应用有哪些功能？
RQ2: What AI algorithms are used to achieve those functions in online higher education?
问题 2：在线高等教育中使用哪些人工智能算法来实现这些功能？
RQ3: What are the effects and implications of AI applications on the instruction and learning processes in online higher education?
问题 3：人工智能应用对在线高等教育的教学过程有哪些影响？

3 Methodology 3 方法

The systematic review methodology used in this study is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) principles, which consists of a 27 -item checklist and a four-phase flow diagram (Moher et al., 2009). The following section will introduce the systematic review procedures.
本研究采用的系统综述方法基于系统综述和元分析首选报告项目（Preferred Reporting Items for Systematic Reviews and Meta-Analyses，PRISMA）原则，该原则包括 27 项检查表和四阶段流程图（Moher 等人，2009 年）。下文将介绍系统性综述的程序。
In order to locate the relevant articles, the systematic search was conducted on the following electronic databases: Web of Science, Scopus, ACM, IEEE, Taylor & Francis, Wiley, EBSCO. We selected these databases since they were considered as the major publisher databases (Guan et al., 2020). Filters were limited to the time period from January 2011 to December 2020 and were applied to the peer-reviewed and empirical research articles written in English in order to ensure the quality of the review articles. After the screening of the full articles, the snowballing approach was performed based on the guidelines to find the articles that were not extracted by using the search strings (Wohlin, 2014). At this stage, Google Scholar was used to searching specific articles.
为了找到相关文章，我们在以下电子数据库中进行了系统搜索：Web of Science、Scopus、ACM、IEEE、Taylor & Francis、Wiley、EBSCO。我们选择这些数据库是因为它们被认为是主要的出版商数据库（Guan 等人，2020 年）。筛选仅限于 2011 年 1 月至 2020 年 12 月期间，并适用于以英语撰写的经同行评审的实证研究文章，以确保综述文章的质量。筛选完整文章后，根据指南采用 "滚雪球 "的方法，找到未通过搜索字符串提取的文章（Wohlin，2014 年）。在此阶段，使用谷歌学术搜索特定文章。

3.2 Search terms 3.2 搜索条件

A structured search strategy was used for various bibliographic databases with keywords used according to each database's specific requirements. In both the electronic and manual searches, specific keywords related to AI and commonly-used algorithms or techniques (i.e., "intelligence", "AI", and "AIEd"), AI applications (i.e., "intelligent tutoring system", "expert system", and "prediction model"), and algorithms (i.e., "decision tree", "machine learning", "neural network", "deep learning", "k-means", "random forest", "support vector machines", "logistic regression", "fuzzy-logic", "Bayesian network", "latent Dirichlet allocation", "natural language processing", "genetic algorithm", and "genetic programming") were used. In addition, as this review focused on the context in online higher education, the following keywords were added: "online education", "online learning", "e-learning", "MOOC", "SPOC" "blended learning", "higher education", "online higher education", "undergraduate education", and "graduated education".
对各种书目数据库采用了结构化检索策略，并根据每个数据库的具体要求使用关键词。在电子检索和人工检索中，与人工智能和常用算法或技术（即 "智能"、"AI "和 "AIEd"）、人工智能应用（即 "智能辅导系统"、"专家系统 "和 "预测模型"）和算法（即、"决策树"、"机器学习"、"神经网络"、"深度学习"、"k-means"、"随机森林"、"支持向量机"、"逻辑回归"、"模糊逻辑"、"贝叶斯网络"、"潜在 Dirichlet 分配"、"自然语言处理"、"遗传算法 "和 "遗传编程"）。此外，由于本综述侧重于在线高等教育，因此添加了以下关键词："在线教育"、"在线学习"、"电子学习"、"MOOC"、"SPOC"、"混合式学习"、"高等教育"、"在线高等教育"、"本科教育 "和 "研究生教育"。
Table 1 A summary of the inclusion and exclusion criteria
表 1 纳入和排除标准摘要
Inclusion criteria 纳入标准 Exclusion criteria 排除标准

研究必须在在线高等教育中进行。
Research must be conducted in online higher educa-a
tion.

会议论文集、书籍章节、杂志、新闻和海报中的研究不包括在内。
Research from conference proceedings, book
chapters, magazines, news, and posters are
excluded.

研究必须报告在人工智能支持下实际教学和学习过程的效果。
Research must report the effects of actual instruction
and learning processes with the support of AI.

未完成的研究不包括在内，例如，只报告人工智能应用设计但不报告实证结果的研究。
Uncompleted research is excluded, for example,
research that only reports AI application design,
but do not report empirical results.

研究成果必须在同行评审期刊上发表。
Research must be published in peer-reviewed
journals.

仅使用自我报告数据收集（如访谈或调查）的实证研究不包括在内。
Empirical research merely used self-report data
collections, such as interviews or surveys, are
excluded.

研究必须以实证研究的形式报告，以展示人工智能在教育领域的实际效果。
Research must reported as empirical research to
demonstrate actual effects of AI in education
contexts.
Research must be reported in English.
研究报告必须用英语撰写。
Research must be published from 2011 to 2020.
研究成果必须在 2011 年至 2020 年期间发表。
Full-text is available. 可提供全文。

3.3 Inclusion and exclusion criteria
3.3 纳入和排除标准

The search criteria were designed to locate the articles that focused on the applications of in online higher education. In terms of the research questions, a set of inclusion and exclusion criteria were adopted (see Table 1).
搜索标准旨在找到有关 在在线高等教育中应用的文章。根据研究问题，采用了一套纳入和排除标准（见表 1）。

3.4 The screening process
3.4 筛选过程

The screening process involved the following steps: (1) removing the duplicated articles, (2) removing the articles that did not meet the inclusion criteria based on the titles and abstracts, (3) reading the full texts again and removing the articles that did not meet the inclusion criteria, (4) using the snowballing approach to further locate the articles in Google Scholar, and (5) extracting data from the final filtered articles (see Fig. 2). All articles were imported into the Mendeley software for screening.
筛选过程包括以下步骤(1) 删除重复文章；(2) 根据标题和摘要删除不符合纳入标准的文章；(3) 再次阅读全文并删除不符合纳入标准的文章；(4) 使用滚雪球法在谷歌学术中进一步查找文章；(5) 从最终筛选出的文章中提取数据（见图 2）。所有文章都被导入到 Mendeley 软件中进行筛选。
The search produced 434 articles from the previously-used search terms, including 92 duplicates that were deleted. By reviewing the titles and abstracts, the number of articles was reduced to 91 based on the criteria (see Table 1). The selected articles were examined by the second author to determine whether they were suitable for the purpose of the study. The first author independently reviewed approximately of the articles to confirm the reliability. The interrater agreement was initially and then was brought to agreement after discussion. Then, the full-text of articles were reviewed by both authors to verify that the articles met all the criteria for inclusion in the review. Eventually, a total of 32 articles that met the criteria were included in the final systematic review.
根据之前使用的检索词，共检索到 434 篇文章，其中删除了 92 篇重复文章。通过审查标题和摘要，根据标准（见表 1）将文章数量减少到 91 篇。第二作者对所选文章进行了审查，以确定它们是否适合本研究的目的。第一作者独立审阅了约 篇文章，以确认其可靠性。最初， ，经过讨论后， 。然后，两位作者对文章的全文进行了审阅，以核实文章是否符合纳入综述的所有标准。最终，共有 32 篇符合标准的文章被纳入最终的系统综述。
Fig. 2 PRISMA flow chart of study selection process
图 2 PRISMA 研究筛选流程图

3.5 Analysis 3.5 分析

The articles that met the inclusion criteria were analyzed using the bibliometric analysis approach (Neuendorf & Kumar, 2015). We calculated the frequencies for each category of AIEd in online higher education. The qualitative content analysis method was carried out to categorize the articles (Zupic & Čater, 2015). We classified the information from the articles relevant to the research questions. Three strategies were used to establish the credibility of the analysis. First, two researchers had ongoing meetings to verify the categories of the reviewed articles (Graneheim & Lundman, 2004). Second, detailed explanations of the categories that emerged as findings for each research question were provided in the result section (Hsieh & Shannon, 2005). Finally, we provided examples to demonstrate
我们采用文献计量分析方法（Neuendorf & Kumar, 2015）对符合纳入标准的文章进行了分析。我们计算了在线高等教育中 AIEd 各类别的频率。我们采用定性内容分析法对文章进行分类（Zupic & Čater, 2015）。我们对文章中与研究问题相关的信息进行了分类。我们采用了三种策略来建立分析的可信度。首先，两名研究人员不断举行会议，以核实所审查文章的类别（Graneheim & Lundman, 2004）。其次，我们在结果部分详细解释了每个研究问题的结论类别（Hsieh & Shannon, 2005）。最后，我们举例说明
how well the categories represented the data to answer the research questions (Graneheim & Lundman, 2004).
这些类别在多大程度上代表了数据，从而回答了研究问题（Graneheim & Lundman, 2004）。

4 Results 4 项成果

Among the 32 empirical articles, of the articles were published after 2016. The articles finally selected were published in 23 different journals. The major countries or areas for the 32 studies were also identified. The most prolific country or area in AIEd research was Spain that had 5 publications ( ), followed by the USA (4 publications, 13%), and Taiwan (4 publications, 13%). Furthermore, three journals were found with more than two relevant articles that met the criteria, including Computers in Human Behavior ( ), Computers & Education , and Interactive Learning Environments (see Appendix Table 2).
在 32 篇实证文章中， ，这些文章发表于 2016 年之后。最终入选的文章发表在 23 种不同的期刊上。此外，还确定了 32 项研究的主要国家或地区。人工智能教育研究最多的国家或地区是西班牙，共发表了 5 篇文章（ ），其次是美国（发表了 4 篇文章，占 13%）和台湾（发表了 4 篇文章，占 13%）。此外，还发现有三份期刊发表了两篇以上符合标准的相关文章，包括《Computers in Human Behavior》（ ）、《Computers & Education》（ ）和《Interactive Learning Environments》（ ）（见附录表 2）。

4.1 RQ1: What are the functions of Al applications in online higher education?
4.1 问题 1：Al 应用软件在在线高等教育中有哪些功能？

There are four major functions of AI applications in online higher education: predictions of learning status, performance or satisfaction , resource recommendation ( ), automatic assessment , and improvement of learning experience ( ) (see Fig. 3).
人工智能在在线高等教育中的应用有四大功能：预测学习状态、成绩或满意度 ，资源推荐 ( ) ，自动评估 ，以及改善学习体验 ( ) （见图 3）。
Fig. 3 The pie chart of the functions of AI applications in online higher education
图 3 在线高等教育中的人工智能应用功能饼图

4.1.1 Predictions of learning status, performance or satisfaction
4.1.1 预测学习状况、成绩或满意度

The first function of AI application is the prediction of student performances, that is to illustrate student learning status or performance in advance. Among 32 reviewed articles, 21 articles focused on the predictions of student performance in online higher education context. Further examinations identified three categories: prediction of dropout risks ( ), prediction of student academic performances ( ), and prediction of student satisfactions about online courses (see Appendix Table 2).
人工智能应用的第一个功能是预测学生成绩，即提前说明学生的学习状况或表现。在 32 篇综述文章中， ，有 21 篇侧重于在线高等教育背景下的学生成绩预测。进一步研究发现有三类：辍学风险预测（ ）、学生学业成绩预测（ ）和学生对在线课程的满意度预测 （见附录表 2）。
In the first category regarding the prediction models for diagnosing the risks of student dropout, Mubarak et al. (2020) constructed a prediction model to predict the students who were at-risk of dropout based on the interaction logs in the online learning environment. The results showed that the proposed models achieved an accuracy of , which was better than the baseline of machine learning models. Aguiar et al. (2014) analyzed engineering students' electronic portfolios to predict their persistence in online courses, and results proved a consistently better performance than those models based on the traditional academic data (SAT scores, GPA, demographics, etc.) alone. The second category is the prediction of student academic performance. For example, Almeda et al. (2018) used the classification models to predict whether students would succeed in the online courses and further used the regression models to predict students' numerical scores. One key finding was that the features related to course comments were significant predictors of the final grades. Romero et al. (2013b) collected the forum messages to predict student performance and they found that the students who actively participated in the forum, posted messages more frequently with a high quality were more likely to pass the course. The third category is the prediction of student satisfaction levels about online courses. Only one study was located: Hew et al. (2020) analyzed the course features of 249 randomly sampled MOOCs and 6,393 students' perceptions were examined to understand what factors predicted student satisfaction. They found that the course instructor, content, assessment, and time schedule played significant roles in explaining the student satisfaction levels.
在第一类关于诊断学生辍学风险的预测模型中，Mubarak 等人（2020）构建了一个预测模型，根据在线学习环境中的交互日志预测有辍学风险的学生。结果表明，所提出的模型准确率达到 ，优于机器学习模型的基线。Aguiar 等人（2014 年）分析了工程专业学生的电子作品集，以预测他们在在线课程中的坚持情况，结果证明，与那些仅基于传统学业数据（SAT 分数、GPA、人口统计学等）的模型相比，这些模型的性能一直较好。第二类是对学生学业成绩的预测。例如，Almeda 等人（2018 年）使用分类模型预测学生是否会在在线课程中取得成功，并进一步使用回归模型预测学生的数字分数。其中一个重要发现是，与课程评论相关的特征是最终成绩的重要预测因素。罗梅罗等人（2013b）收集了论坛留言来预测学生成绩，他们发现，积极参与论坛、更频繁地发布高质量留言的学生更有可能通过课程。第三类是预测学生对在线课程的满意度。只找到了一项研究：Hew等人（2020）分析了随机抽样的249门MOOC的课程特点，并考察了6393名学生的看法，以了解哪些因素能预测学生的满意度。他们发现，课程教师、内容、评估和时间安排在解释学生满意度方面发挥了重要作用。

4.1.2 Resource recommendation
4.1.2 资源建议

Among 32 reviewed articles, 7 articles focused on the resource recommendation in online higher education context. For example, Benhamdi et al. (2017) designed a recommendation approach to provide online students with the appropriate learning materials based on their preferences, interests, background knowledge, and memory capacities to store information. The results concluded that this recommendation system improved students' learning quality. Christudas et al. (2018) used the compatible genetic algorithm (CGA) to provide suitable learning content for individual students based on the preferred learning objects they previously chose. The results showed that the students' scores and satisfaction levels were improved in an e-learning environment. In the online programming courses, Cárdenas-Cobo et al. (2020) developed a system called CARAMBA to suggest suitable learning exercises for students in scratch programming. Results confirmed that those exercise
在32篇综述文章中，有7篇 ，重点关注在线高等教育背景下的资源推荐。例如，Benhamdi 等人（2017）设计了一种推荐方法，根据在线学生的偏好、兴趣、背景知识和存储信息的记忆能力，为他们提供合适的学习材料。结果认为，这种推荐系统提高了学生的学习质量。Christudas 等人（2018）使用兼容遗传算法（CGA），根据学生之前选择的偏好学习对象，为学生个人提供合适的学习内容。结果表明，在网络学习环境中，学生的分数和满意度都得到了提高。在在线编程课程中，Cárdenas-Cobo 等人（2020 年）开发了一个名为 CARAMBA 的系统，为学生推荐合适的从头开始编程学习练习。结果证实，这些练习
recommendations in Scratch had improved the student's programming capabilities. In summary, AI has been used in online higher education to recommend learners suitable and personalized resources based on learners' fixed and dynamic characteristics.
Scratch 中的建议提高了学生的编程能力。总之，人工智能已被用于在线高等教育，根据学习者的固定和动态特征，为学习者推荐合适的个性化资源。

4.1.3 Automatic assessment
4.1.3 自动评估

Among 32 reviewed articles, 2 articles ( ) focused on the automatic assessment in online higher education context. Hooshyar et al. (2016) developed an ITS system called Tic-tac-toe Quiz for Single-Player (TRIS-Q-SP) to provide students formative assessment of their computer programming performances and problem-solving capacities. The empirical research demonstrated that the proposed system enhanced student learning interests, positive attitudes, degree of technology acceptance, and problem solving activities. Aluthman (2016) developed the automated essay evaluation (AEE) system to provide students with immediate assessment, feedback, and automated scores in an online English learning environment and examined the effects of utilizing AEE on undergraduate students' writing performance. The results indicated that the AEE system had a positive effect on improving students' writing performance. In summary, AI has been used in online higher education to automatically assess students' performances and learning capacities, to provide timely feedback to students, and to improve students' self-awareness and self-reflection.
在32篇综述文章中，有2篇文章（ ）关注在线高等教育背景下的自动评估。Hooshyar等人（2016）开发了一个名为 "单人井字测验"（TRIS-Q-SP）的ITS系统，对学生的计算机编程表现和解决问题的能力进行形成性评估。实证研究表明，所提议的系统提高了学生的学习兴趣、积极态度、技术接受程度和问题解决活动。Aluthman （2016）开发了自动作文评价（AEE）系统，在在线英语学习环境中为学生提供即时评估、反馈和自动评分，并考察了使用 AEE 对本科生写作成绩的影响。结果表明，AEE 系统对提高学生的写作成绩有积极作用。总之，人工智能已被应用于在线高等教育，以自动评估学生的表现和学习能力，为学生提供及时反馈，并提高学生的自我意识和自我反思能力。

4.1.4 Improvement of learning experience
4.1.4 改善学习体验

Among 32 reviewed articles, 2 articles ( ) focused on the optimization of learning experiences by improving learner interactions with learning environments or resources in online higher education context. Ijaz et al. (2017) created virtual reality (VR) tool that applied the AI technique for history learning. The VR tool allowed students immerse themselves into the virtual environments of cities and learn by browsing and interacting with virtual citizens. The results confirmed that this AIenabled virtual learning mode was more engaging and motivating for students, compared to simply reading the history texts or watching educational videos. KoćJanuchta et al. (2020) compared student interaction and learning quality between the designed AI-enriched biology books and traditional e-books. Results found that the students who used the AI-enriched book asked more questions and kept higher retention than those engaged in reading the traditional e-books.
在32篇综述文章中，有2篇文章（ ）侧重于通过改善在线高等教育背景下学习者与学习环境或资源的交互来优化学习体验。Ijaz 等人（2017）创建了虚拟现实（VR）工具，将人工智能技术应用于历史学习。该虚拟现实工具让学生沉浸在城市的虚拟环境中，通过浏览和与虚拟市民互动来学习。研究结果证实，与单纯阅读历史课本或观看教育视频相比，这种人工智能虚拟学习模式更能吸引和激励学生。KoćJanuchta 等人（2020 年）比较了所设计的人工智能强化生物书籍与传统电子书籍之间的学生互动和学习质量。结果发现，与阅读传统电子书籍的学生相比，使用人工智能强化书籍的学生提出的问题更多，保持率更高。

4.2 RQ2: What Al algorithms are used to achieve those functions in online higher education?
4.2 问题 2：在线高等教育中使用哪些 Al 算法来实现这些功能？

Among the 32 reviewed articles, 24 articles specified the AI algorithms they used in the research. Those eight articles used AI systems or tools (e.g., recommendation systems) but did not specify the AI algorithms they used. Among the 24 articles, the most commonly used AI algorithms were DT ( ), NN , NB
在 32 篇被审查的文章中，有 24 篇文章明确说明了他们在研究中使用的人工智能算法。这 8 篇文章使用了人工智能系统或工具（如推荐系统），但没有说明所使用的人工智能算法。在这 24 篇文章中，最常用的人工智能算法是 DT ( ), NN , NB
Fig. 4 The distribution of AI algorithms. Note: DT: Decision Tree; NN: Neural Network; SVM: Support Vector Machine; NB: Naive Bayes; RF: Random Forest; LGR: Logistic Regression; LR: Liner Regression; KNN: K-Nearest Neighbours; NLP: Neural Language Processing; BN: Bayes Network; XGBoost: Extreme Gradient Boosting; SVC: Support Vector Classification; Splines: Multivariate Adaptive Regression Splines; SMO: Sequential Minimal Optimizer; RG: Regression; LDA: Linear Discriminant Analysis; IOHMM: Input-Output Hidden Markov Model; GaNB: Gaussian Naive Bayes; GA: Genetic Algorithm; CART: Classification and Regression Tree; MLP: Multi-Layer Perceptron; BART: Bayesian Additive Regressive Trees
图 4 人工智能算法的分布。注：DT：决策树；NN：神经网络；SVM：NB: Naive Bayes; RF: Random Forest; LGR: Logistic Regression; LR: Liner Regression; KNN：KNN: K-Nearest Neighbours; NLP: Neural Language Processing; BN: Bayes Network; XGBoost：Extreme Gradient Boosting（极梯度提升）；SVC：支持向量分类；Splines：SMO：顺序最小优化器；RG：回归；LDA：线性判别分析：线性判别分析；IOHMM：输入输出隐马尔可夫模型；GaNB：高斯直觉贝叶斯；GA：GA：遗传算法；CART：分类和回归树；MLP：多层感知器；BART：贝叶斯加性回归树
and SVM . Some research used multiple algorithms in one study (see Fig. 4 and Appendix Table 3).
和 SVM 。有些研究在一项研究中使用了多种算法（见图 4 和附录表 3）。
The systematic review found that relatively traditional algorithms included DT, LGR, NB, SVM, and NLP. For example, Almeda et al. (2018) used the J48 DT to predict whether a student would successfully pass the course and further used the regression models to predict student's numerical grades. Mubarak et al. (2020) proposed two AI models, namely the Logistic Regression and the input-output hidden Markov model, to predict student dropout risk. Yoo and Kim (2014) used online discussion participation as the predictor for class project performance and used Support Vector Machine (SVM) for data processing and automatic classifiers. Helal et al. (2018) created different classification models for predicting student performance, including two black-box methods, namely NB and SMO, an optimization algorithm for training SVM, and two white box methods (i.e., J48 and JRip). Natural language processing (NLP) was applied to automatically assess student performance and to detect student satisfaction. For example, Aluthman (2016) adopted the NLP techniques to automatically evaluate essays and provided students with both automated scores and immediate feedback. Hew et al. (2020) used the NLP techniques to identify what students commented to predict students' satisfaction levels with MOOCs.
系统回顾发现，相对传统的算法包括 DT、LGR、NB、SVM 和 NLP。例如，Almeda 等人（2018）使用 J48 DT 预测学生是否能顺利通过课程，并进一步使用回归模型预测学生的数字成绩。Mubarak 等人（2020）提出了两种人工智能模型，即逻辑回归和输入输出隐马尔科夫模型，用于预测学生辍学风险。Yoo 和 Kim（2014）将在线讨论参与度作为班级项目成绩的预测指标，并使用支持向量机（SVM）进行数据处理和自动分类。Helal 等人（2018）创建了不同的分类模型来预测学生成绩，包括两种黑盒方法，即 NB 和 SMO，一种用于训练 SVM 的优化算法，以及两种白盒方法（即 J48 和 JRip）。自然语言处理（NLP）被用于自动评估学生成绩和检测学生满意度。例如，Aluthman（2016）采用 NLP 技术自动评估论文，并为学生提供自动评分和即时反馈。Hew 等人（2020 年）使用 NLP 技术来识别学生的评论内容，从而预测学生对 MOOC 的满意程度。
Advanced machine learning algorithms such as NN and GA were also used in some research. For example, Sukhbaatar et al. (2019) employed the NN method
一些研究还使用了先进的机器学习算法，如 NN 和 GA。例如，Sukhbaatar 等人（2019 年）采用了 NN 方法
to predict student failure tendency based on multiple variables extracted from the online learning activities in a learning management system. Results showed that of the failing students were correctly identified after the first quiz submission; after the mid-term examination, of the failing students were correctly predicted. Yang et al. (2017) presented a time series NN method for predicting the evolution of student average CFA grade in two MOOCs. Results found that the -based algorithms consistently outperformed a baseline model that simply averaged historical CFA data. Christudas et al. (2018) presented a GA-enable approach for recommending personalized learning content for individual students in the e-learning system and the results found an improvement of students' final scores in the course.
根据从学习管理系统的在线学习活动中提取的多个变量，预测学生的不及格倾向。结果表明，在第一次提交测验后， ，正确识别了不及格学生；在期中考试后， ，正确预测了不及格学生。Yang等人（2017）提出了一种时间序列NN方法，用于预测两个MOOC中学生平均CFA成绩的变化。结果发现，基于 的算法始终优于简单平均历史 CFA 数据的基线模型。Christudas 等人（2018）提出了一种可启用 GA 的方法，用于在电子学习系统中为学生个人推荐个性化学习内容，结果发现学生的课程最终成绩有所提高。
An important factor in the AI models is the choice of input variables. Primary variables used in the 32 articles included demographic, assignment, previous score, quiz access, forum access, course material access, login behavior, etc. (see Appendix Table 3). Demographic variables include students' gender, age, race, ethnicity information. The assignment category includes whether the assignment is completed and submitted. The previous score refers to the historical grades of students for different courses or learning activities, the standardized high school test scores or continuous assessment activities. The quiz access includes the times of quiz attempt, whether the quiz is complete or not, quiz scores, and solving time. The forum is where student's ideas on a particular subject have exchange and communicated with their instructor and peers. This category includes the number of forum views, number of forum posts, times of reply, post content, etc. The course material information contains the total time material viewed, and time consumed on materials. Login behaviors mainly include the online learning days, times of access, time consumption per week. Among those variables, forum and course material access were the most commonly used, followed by student scores and login behaviors.
人工智能模型的一个重要因素是输入变量的选择。32 篇文章中使用的主要变量包括人口统计学、作业、以前的分数、测验访问、论坛访问、课程材料访问、登录行为等（见附录表 3）。人口统计学变量包括学生的性别、年龄、种族和民族信息。作业类别包括作业是否完成和提交。以往成绩是指学生在不同课程或学习活动中的历史成绩、高中标准化考试成绩或连续评估活动的成绩。测验访问包括测验尝试时间、测验是否完成、测验分数和解题时间。论坛是学生就某一主题与教师和同学交流和沟通想法的地方。这类信息包括论坛浏览次数、论坛帖子数、回复次数、帖子内容等。课程资料信息包括资料浏览总时间和资料消耗时间。登录行为主要包括在线学习天数、登录次数、每周耗时等。在这些变量中，最常用的是论坛和教材访问，其次是学生分数和登录行为。
When using AI algorithms, researchers tended to compare the efficiency and effectiveness of using different algorithms to address the same research purpose. For example, Moreno-Marcos et al. (2018) collected the data from a Java programming MOOC to determine what factors affected the predictions and in which way it was possible to predict scores. In this work, four algorithms, namely RG, SVM, DT, and RF, were used and results were compared to identify which one provided the best results. In a blended learning context, Sukhbaatar et al. (2019) proposed an early prediction scheme to identify the student who was at risk of failing. NN, SVM, DT, and NB methods were compared for the failure prediction in terms of prediction effectiveness. Baneres et al. (2019) presented an adaptive predictive model called GRADUAL AT-RISK MODEL and developed an early warning system, and an early feedback prediction system to intervene at-risk identification of students. Four classification algorithms, NB, CART DT, KNN, and SV, were tested to determine which classification algorithm best fit the GAR model. In addition, Howard et al. (2018) examined eight prediction methods, including BART, RF, PCR, Splines, KNN, NN, and SVM, to identify students' final grades. Huang et al. (2020) collected the applied eight classifiers based on students' online learning logs, namely GaNB, SVC, linear-SVC, LR, DT, RF, NN, and XGBoost, to predict the student academic performances. They also employed five evaluators, namely accuracy,
在使用人工智能算法时，研究人员倾向于比较使用不同 ，以达到相同的研究目的。例如，Moreno-Marcos 等人（2018 年）收集了 Java 编程 MOOC 的数据，以确定哪些因素会影响预测结果，以及以何种方式可以预测分数。在这项工作中，使用了四种算法，即 RG、SVM、DT 和 RF，并对结果进行了比较，以确定哪种算法提供的结果最好。在混合式学习背景下，Sukhbaatar 等人（2019）提出了一种早期预测方案，以识别有不及格风险的学生。他们比较了 NN、SVM、DT 和 NB 方法在失败预测方面的预测效果。Baneres等人（2019）提出了一种名为 "GRADUAL AT-RISK MODEL "的自适应预测模型，并开发了一个早期预警系统和一个早期反馈预测系统来干预高危学生的识别。测试了 NB、CART DT、KNN 和 SV 四种分类算法，以确定哪种分类算法最适合 GAR 模型。此外，Howard 等人（2018）研究了八种预测方法，包括 BART、RF、PCR、Splines、KNN、NN 和 SVM，以识别学生的最终成绩。Huang 等人（2020）收集了基于学生在线学习日志的八种分类器，即 GaNB、SVC、linear-SVC、LR、DT、RF、NN 和 XGBoost，用于预测学生的学习成绩。他们还采用了五种评价指标，即准确率、
Fig. 5 The pie chart of the implications of AI applications in online higher education
图 5 人工智能应用对在线高等教育的影响饼图
precision, recall, the F1-measure, and AUC to measure the predictive performance for classification methods.
精确度、召回率、F1 测量值和 AUC 用来衡量分类方法的预测性能。

4.3 RQ3: What are the effects and implications of Al applications on the instruction and learning processes in online higher education?
4.3 问题 3：Al 应用程序对在线高等教育的教学过程有哪些影响？

The positive effects were identified by using AI applications in online higher education to improve the instruction and learning quality: a high quality of AI-enabled prediction with multiple input variables , a high quality of AI-enabled recommendations based on student characteristics , an improvement of students' academic performances , an improvement of online engagement and participation ( ) (see Fig. 5 and Appendix Table 4).
通过在在线高等教育中使用人工智能应用来提高教学和学习质量，发现了积极的效果：人工智能支持的多输入变量高质量预测 ，人工智能支持的基于学生特征的高质量推荐 ，学生学业成绩的提高 ，在线参与度和参与度的提高( )（见图 5 和附录表 4）。

4.3.1 A high quality of AI-enabled prediction with multiple input variables
4.3.1 多输入变量的高质量人工智能预测

Evidence has been reported that students enrolled in online courses have higher dropout rates than those in the traditional classroom settings (Breslow et al., 2013; Tyler-Smith, 2006). An increase in rates of dropout unavoidably leads to reduce graduation rates, which may have a negative effect on the online learning quality (Simpson, 2018).AI applications in online higher education are mainly the prediction models to predict the student's risk of dropouts and final academic performance. The prediction of student academic performance help identifies the students who have difficulties understanding the course materials or who are at risk of failing the exam (Tomasevic et al., 2020). For example, the results obtained by Baneres et al. (2019) proved that the prediction system achieved early detection of potential at-risk students, offered the guidance and feedback with visualization dashboards, and enhanced the interaction with at-risk students. In this way, the prediction system
有证据表明，与传统课堂教学相比，在线课程的学生辍学率更高（Breslow 等人，2013 年；Tyler-Smith，2006 年）。辍学率的增加不可避免地会导致毕业率的降低，这可能会对在线学习质量产生负面影响（Simpson，2018）。在线高等教育中的人工智能应用主要是预测模型，用于预测学生的辍学风险和最终学业成绩。对学生学业成绩的预测有助于识别在理解课程材料方面存在困难或有考试不及格风险的学生（Tomasevic et al.）例如，Baneres 等人（2019）的研究结果证明，预测系统实现了对潜在高危学生的早期检测，通过可视化仪表盘提供指导和反馈，并加强了与高危学生的互动。因此，预测系统
helped instructors or administrators identify students' learning issues, assist students in regulating and reflecting on the learning processes, and further provide students with instant intervention and guidance at an early stage during the course (MorenoMarcos et al., 2018).
帮助教师或管理人员发现学生的学习问题，协助学生调节和反思学习过程，并进一步在课程的早期阶段为学生提供即时干预和指导（Moreno-Marcos 等人，2018 年）。
Articles reviewed in this work indicated a high quality of accuracy of the prediction models that have used multiple input variables and advanced AI algorithms. For example, Aguiar et al. (2014) found that the performance of the prediction models with ePortfolio data was consistently better than those models based on academic performance data alone. Costa et al. (2017) predicted students' academic failure in introductory programming courses based on multiple student data, including age, gender, student registration, semester, campus, year of enrolling in the course, status on discipline, number of correct exercises, and performance of the students etc. Moreover, advanced AI algorithms such as genetic algorithms and input-output hidden Markov model has been applied in predicting systems, which was proved to achieve more accurate results than traditional algorithms (Mubarak et al., 2020). Therefore, to achieve an accurate prediction, AI-enabled models should first consider using multiple input variables from students' learning processes rather than merely using summative performance scores, and second use advanced AI algorithms to achieve precisions of the relations between the learning inputs and performance outputs (Chassignol et al., 2018; Godwin & Kirn, 2020; Tomasevic et al., 2020).
这项工作所审查的文章表明，使用多种输入变量和先进人工智能算法的预测模型具有很高的准确性。例如，Aguiar 等人（2014 年）发现，使用电子档案袋数据的预测模型的性能始终优于仅基于学习成绩数据的模型。Costa等人（2017）根据学生的多种数据，包括年龄、性别、学籍、学期、校区、选课年份、学科状况、练习正确率、学生成绩等，预测了学生在编程入门课程中的学业失败情况。此外，先进的人工智能算法，如遗传算法和输入输出隐马尔科夫模型，已被应用于预测系统中，事实证明其结果比传统算法更准确（Mubarak 等人，2020 年）。因此，要实现准确预测，人工智能化模型首先应考虑使用学生学习过程中的多个输入变量，而非仅仅使用总结性成绩分数；其次应使用先进的人工智能算法实现学习输入与成绩输出之间关系的精确化（Chassignol等人，2018；Godwin & Kirn，2020；Tomasevic等人，2020）。

4.3.2 A high quality of Al-enabled recommendations based on student characteristics
4.3.2 根据学生特点提供高质量的 Al-enabled 建议

A high quality of recommendation requires the algorithm model take into consideration students' diverse characteristics, such as knowledge levels, learning styles or preferences, learning profiles and interests, etc. Our review showed that five studies related to recommendation have reported that their methods generated a high-quality recommendation for students. For example, Benhamdi et al. (2017) proposed a new recommendation approach based on collaborative, content-based filtering to provide students the best learning materials according to their preferences, interests, background knowledge, and memory capacity. The experiment results showed there was a significant difference between the marks of pre-tests and post-tests, which indicated that students acquired more knowledge when they used the proposed recommender system. Additionally, Bousbahi & Chorfi (2015) used the case-based reasoning (CBR) approach and the special retrieval information technique to recommend the most appropriate MOOCs courses that best-suited student needs based on their learning profiles, needs, and knowledge levels. In addition to personalized learning recommendation. Dwivedi & Bharadwaj (2015) designed an e-Learning recommender system for a group of students by considering students' learning styles, knowledge levels, and ratings of learners in a group. The results demonstrated the effectiveness of the proposed group recommendation strategy. Although those studies verified the short-term effects of recommendation systems, there is a lack of investigations on the effects of applying the recommendation systems or methods in students' long-term learning. Future work should introduce both fixed and dynamic
高质量的推荐要求算法模型考虑到学生的不同特点，如知识水平、学习风格或偏好、学习概况和兴趣等。我们的综述显示，有五项与推荐相关的研究报告称，他们的方法为学生生成了高质量的推荐。例如，Benhamdi 等人（2017）提出了一种基于协作、基于内容过滤的新推荐方法，根据学生的偏好、兴趣、背景知识和记忆能力为其提供最佳学习材料。实验结果表明，前测和后测的分数存在显著差异，这表明学生在使用所提出的推荐系统时掌握了更多的知识。此外，Bousbahi & Chorfi（2015）采用基于案例的推理（CBR）方法和特殊检索信息技术，根据学生的学习情况、需求和知识水平，推荐最适合学生需求的MOOCs课程。除了个性化学习推荐外。Dwivedi & Bharadwaj（2015）通过考虑学生的学习风格、知识水平和组内学习者的评分，为一组学生设计了一个电子学习推荐系统。研究结果证明了所提出的小组推荐策略的有效性。虽然这些研究验证了推荐系统的短期效果，但缺乏对推荐系统或方法在学生长期学习中的应用效果的调查。未来的工作应同时引入固定和动态的
characteristics of students, carry out experiments with large sample size, in order to confirm the accuracy of recommendation systems.
根据学生的特点，进行大样本量的实验，以确认推荐系统的准确性。

4.3.3 An improvement of students' academic performance
4.3.3 提高学生的学习成绩

The results indicated that AI systems and tools helped improve students' academic performances by optimizing learning environments and experiences, recommending learning resources or providing automatic feedback and assessment in online learning. For example. Ijaz et al. (2017) found that students in the VR context combined with AI techniques performed better in comprehending the materials than the control groups without AI support. Cárdenas-Cobo et al. (2020) presented an easy-to-use web application called CARAMBA involving Scratch alongside a recommender system for exercises. Results confirmed that, in terms of pass rates, the recommending exercises in Scratch had a positive effect on the student's programming abilities. The pass rate was over , which was higher than that in the previous exercises with Scratch (without recommendation) and higher than the historical results of traditional programming teaching (without Scratch). Compared to the traditional learning approaches (e.g., reading textbooks), AI can provide students with more intelligent and personalized interaction forms, such that the interaction between students and learning resources and the degree of participation in learning can be improved. More importantly, since improper contents that do not fit students' learning styles, or their knowledge or ability levels may lead to information overload or lack of learning orientation, which would negatively affect student academic performance (Chen, 2008; Christudas et al., 2018). AI can optimized personalized resource recommendations based on students' characteristics, which has been emphasized as a crucial issue in e-learning and online learning (Chang & , 2013). In addition, providing automatic feedback was also a good way to improve student academic performance because it could give students personalized diagnoses and suggestions (e.g., Aluthman, 2016), that improves student's learning motivations and effectiveness (Gardner et al., 2002; Henly, 2003). In conclusion, it can be summarized from the existing research that with the support of AI, student academic performance can be promoted in terms of the final grades, completion rate of course or learning satisfaction levels.
结果表明，人工智能系统和工具通过优化学习环境和体验、推荐学习资源或在在线学习中提供自动反馈和评估，有助于提高学生的学业成绩。例如Ijaz 等人（2017）发现，与没有人工智能支持的对照组相比，结合了人工智能技术的 VR 情境下的学生在理解教材方面表现更好。Cárdenas-Cobo等人（2020年）介绍了一个名为CARAMBA的易用网络应用程序，其中涉及Scratch和练习推荐系统。结果证实，就通过率而言，Scratch 推荐练习对学生的编程能力有积极影响。通过率超过 ，比之前使用 Scratch（无推荐）的练习高出 ，比传统编程教学（无 Scratch）的历史结果高出 。与传统的学习方式（如阅读课本）相比，人工智能可以为学生提供更加智能化和个性化的交互形式，从而提高学生与学习资源之间的互动性和学习的参与度。更重要的是，由于不适合学生学习风格、知识或能力水平的不当内容可能会导致信息超载或缺乏学习导向，从而对学生的学业成绩产生负面影响（Chen，2008；Christudas et al.，2018）。人工智能可以根据学生的特点优化个性化资源推荐，这已被强调为电子学习和在线学习中的一个关键问题（Chang & , 2013）。此外，提供自动反馈也是提高学生学习成绩的好方法，因为它可以给学生提供个性化的诊断和建议（例如，Aluthman，2016），从而提高学生的学习动机和学习效率（Gardner 等人，2002；Henly，2003）。总之，从现有研究中可以总结出，在人工智能的支持下，学生的学业成绩可以在期末成绩、课程完成率或学习满意度等方面得到提升。

4.3.4 An improvement of online engagement and participation
4.3.4 提高在线参与度和参与程度

AI systems or techniques can positively influence student's online engagement through providing personalized resources, automatic assessment, and timely feedback. For example, Ijaz et al. (2017) investigated a technological combination of AIenabled virtual reality with an aim to improve the learning experiences and learner engagement. The results found that compared to simply reading the history texts or watching educational videos, the AI-enabled learning mode was more engaging and motivating for the students. Koć-Januchta et al. (2020) explored students' engagement and patterns of activity with AI book and traditional digital E-book. The research collected students' pre- and post-test scores, cognitive load, motivation, usability questionnaires and interviews. The results found that students who
人工智能系统或技术可以通过提供个性化资源、自动评估和及时反馈，对学生的在线参与产生积极影响。例如，Ijaz 等人（2017 年）研究了人工智能虚拟现实的技术组合，旨在改善学习体验和提高学习者的参与度。结果发现，与单纯阅读历史课文或观看教育视频相比，人工智能支持的学习模式更能吸引学生，更能激发他们的学习兴趣。Koć-Januchta 等人（2020 年）探讨了学生使用人工智能图书和传统数字电子书的参与度和活动模式。研究收集了学生的前后测试得分、认知负荷、学习动机、可用性问卷和访谈。结果发现
used the AI-enriched books asked more questions and kept higher retention than those engaged in reading the traditional e-books, which indicated the improvement of student engagement. Therefore, the results indicate that AI supported learning has potential to improve students' online engagement with learning materials, online courses and their peers. Given that online students often have a low level of participation in online learning, which would lead to problems such as dropping out or academic failure, AI technological support has potential to improve students' online engagement with learning materials, online courses and their peers. This is a great way to improve students' learning engagement and participation and thus reduce their academic failure to some extent.
与阅读传统电子书籍的学生相比，使用人工智能辅助学习的学生提出了更多问题，保留率也更高，这表明学生的参与度有所提高。因此，研究结果表明，人工智能辅助学习有可能提高学生对学习材料、在线课程和同伴的在线参与度。鉴于在线学生的在线学习参与度往往较低，会导致辍学或学业失败等问题，人工智能技术支持有可能提高学生对学习材料、在线课程和同伴的在线参与度。这是提高学生学习投入度和参与度的好方法，从而在一定程度上减少他们的学业失败。

5 Discussions and implications
5 讨论和影响

The application of artificial intelligence (AI) has brought new challenges for improving instruction and learning in online higher education. Given that there is limited literature review examining the actual effects of in online higher education, it is necessary to gain a deep understanding of the functions, effects, and implications of AI applications in online higher education. Furthermore, there has been a critical gap between what AIEd technologies can do, how they are implemented in authentic online higher education settings, and to what extent the use of AI applications influence actual online instruction and learning (Kabudi et al., 2021; Ouyang & Jiao, 2021). The systematic literature review specifically focuses on AI applications in online higher education and the results show that performance prediction, resource recommendation, automatic assessment, and improvement of learning experiences are the four main funcitons of AI applications in online higher education. Regarding AI techniques, it is found that the algorithms such as DT, LRG, NN, and BT, are the most commonly adopted in the online educational contexts. Advanced DL algorithms such as GA and DNN have seldom been found, which is consistent with the findings from Zawacki-Richter et al. (2019) and Chen et al. (2020b). Regarding the actual effects of AI in online higher education, several empirical research have reported positive effects of AI application in improving online instruction and learning quality, including a high quality of AI-enabled prediction, a high quality of AI-enabled recommendations, an improvement of academic performances as well as an improvement of online engagement and participation. Based on the review results, to achieve a high quality of prediction, assessment or recommendation, AI-enabled systems or models should first model take into consideration students' diverse characteristics from both learning processes and summative performances, and second use advanced algorithms to achieve precisions of the outcome in order to improve students' learning motivation, engagement and performance. With the innovation and advancement of AI technologies and techniques, the applications of AI promote the transformation of higher education from traditional, instructor-directed lecturing to AI-enabled, student-centered learning (Chen et al., 2020a; Ouyang & Jiao, 2021).
人工智能（AI）的应用为改善在线高等教育的教学和学习带来了新的挑战。鉴于研究 在在线高等教育中实际效果的文献综述有限，有必要深入了解在线高等教育中人工智能应用的功能、效果和影响。此外，在人工智能教育技术能做什么、如何在真实的在线高等教育环境中实施，以及人工智能应用的使用在多大程度上影响了实际的在线教学和学习之间，一直存在着严重的差距（卡布迪等人，2021；欧阳和焦，2021）。系统性文献综述特别关注在线高等教育中的人工智能应用，结果表明，成绩预测、资源推荐、自动评估和改善学习体验是在线高等教育中人工智能应用的四大功能。在人工智能技术方面，研究发现 DT、LRG、NN 和 BT 等算法是在线教育中最常采用的算法。GA和DNN等高级DL算法很少被发现，这与Zawacki-Richter等人（2019）和Chen等人（2020b）的研究结果一致。关于人工智能在在线高等教育中的实际效果，一些实证研究报道了人工智能应用在提高在线教学和学习质量方面的积极作用，包括人工智能支持的高质量预测、人工智能支持的高质量推荐、学业成绩的提高以及在线参与度和参与度的提高。根据综述结果，要实现高质量的预测、评估或推荐，人工智能系统或模型首先应建立模型，从学习过程和总结性表现两方面考虑学生的不同特征，其次使用先进的 算法实现结果的精确性，以提高学生的学习动力、参与度和学习成绩。随着人工智能技术和工艺的创新与进步，人工智能的应用促进了高等教育从传统的、以教师为主导的讲授向人工智能支持的、以学生为中心的学习转变（陈等，2020a；欧阳和焦，2021）。
Based on the results of this literature review, we propose the theoretical, technological, and practical implications for the applications of AI in online higher
根据文献综述的结果，我们提出了人工智能在在线高等教育中应用的理论、技术和实践意义。
education. First, from the theoretical perspective, educational theories have not been adopted to underpin the application of in online higher education. Similar to the previous work (Chen et al., 2020b, Ouyang & Jiao, 2021; Zawacki-Richter et al., 2019), few studies have focused on building the connection between the educational and learning theories and AI-supported online higher education. Although advanced AI technology has the potential to improve online higher education quality (Holmes et al., 2019), good educational outcomes do not occur by merely using advanced AI technologies (Castañeda & Selwyn, 2018; Du Boulay, 2000; Selwyn, 2016). More importantly, the use of AI technologies and applications generally imply different pedagogical perspectives, which in turn pose critical influences on the design and implementation of instruction and learning (Hwang et al., 2020; Ouyang & Jiao, 2021). As Chen et al. (2020b) suggested, social constructivism (Vygotsky, 1978), situational theory (Kim & Grunig, 2011), distributed cognition (Hollan et al., 2000) are worthwhile to be studied while integrating AI applications in online higher education. Ouyang & Jiao (2021) proposed the three paradigms of AIEd from the theoretical perspective (i.e., AI-directed, learner-as-recipient, AI-supported, learner-as-collaborator, and AI-empowered, learner-as-leader), which could serve as the reference framework to explore varied ways of addressing the learning and instructional issues with the AI applications. In summary, based on the existing educational and learning theories, researchers and practitioners can integrate pedagogy and learning sciences with AIEd applications to derive multiple perspectives and interpretations about AIEd in online higher education (Hwang et al., 2020; Hwang Tu, 2021; Ouyang & Jiao, 2021).
教育。首先，从理论角度来看，尚未采用教育理论来支撑 在在线高等教育中的应用。与之前的研究（Chen et al., 2020b, Ouyang & Jiao, 2021; Zawacki-Richter et al., 2019）类似，很少有研究关注建立教育和学习理论与人工智能支持的在线高等教育之间的联系。尽管先进的人工智能技术具有提高在线高等教育质量的潜力（Holmes 等，2019），但良好的教育成果并不是仅仅通过使用先进的人工智能技术就能实现的（Castañeda & Selwyn，2018；Du Boulay，2000；Selwyn，2016）。更重要的是，人工智能技术和应用的使用通常意味着不同的教学视角，这反过来又对教学和学习的设计与实施产生关键影响（Hwang 等人，2020；欧阳和焦，2021）。正如 Chen 等人（2020b）所提出的，在在线高等教育中整合人工智能应用时，社会建构主义（Vygotsky，1978 年）、情境理论（Kim & Grunig，2011 年）、分布式认知（Hollan 等人，2000 年）值得研究。欧阳和焦（2021）从理论角度提出了人工智能教育的三种范式（即人工智能指导下的学习者即接受者、人工智能支持下的学习者即合作者和人工智能赋能下的学习者即领导者），可作为探索人工智能应用解决学习和教学问题的多种途径的参考框架。总之，基于现有的教育和学习理论，研究者和实践者可以将教学法和学习科学与人工智能教育应用相结合，对在线高等教育中的人工智能教育进行多角度、多层面的解读（Hwang 等，2020；Hwang Tu，2021；Ouyang & Jiao，2021）。
Second, from the technological perspective, AI technologies, models and applications in online higher education are expected to seek the potential of integrating students' learning process characteristics with AI, to connect and strengthen interactions between AI and educators and students, and to address the issues regarding the biases in AI algorithms, and non-transparency of why and how AI decisions are made (Hwang et al., 2020; Hwang & Tu, 2021; Ouyang & Jiao, 2021). This review has illustrated the importance of data collection and analysis of learning process data in addition to summative data in order to achieve a high quality of AI-enabled prediction or recommendation. The advancement of emerging computer technologies, such as quantum computing, wearable devices, robot control, and sensing devices, and wireless communication technologies, have provided new affordances and opportunities to integrate AI with collection and analysis of online learning processes in online higher education (Chen et al., 2020b; Hwang et al., 2020). When integrating into online higher education, AI has the potential to provide student with a practical or experiential learning experience, particularly when is integrated with other technologies such as VR, 3D, gaming, and simulation, and thereby improving the student learning experience and academic performance. To advance the state-of-the-art of AIEd technologies in online higher education, it is necessary to provide a bridge to facilitate the interaction and collaboration between educators or students with AI systems or tools, which can help obtain a multifaceted understanding of student status and achieve a good learning performance prediction (Giannakos et al., 2019). With the support of real-time AI algorithm models, it has the potential to collect and feedback information from human 
to AI systems in a timely fashion. In this way, AI applications can collect and make sense of the user-generated data to provide a deeper understanding of the real-time interaction between humans and technologies in online higher education (Giannakos et al., 2019; Ouyang & Jiao, 2021; Xie et al., 2019). Future research can consider to develop prediction models that can be used in heterogeneous contexts across platforms, thematic areas, and course durations, This approach has potential to enhance the predictive power of current models by improving algorithms or adding novel higher-order features from students or groups (Moreno-Marcos et al., 2019). Overall, since online higher education stresses learner-centered learning, integration of human intelligence and machine intelligence can help AIEd transform from traditional lecturing to learner-centered learning (Ouyang & Jiao, 2021). 
Third, from the practical perspective, AIEd advancement calls for more empirical research to examine what are the different roles of in online higher education, how AI are connected to the existing educational and learning theories, and to what extent the use of technologies influence online learning quality (Hwang et al., 2020; Kabudi et al., 2021; Ouyang & Jiao, 2021). As researchers pointed out in a recent literature review, there has been a discrepancy between the potentials of AIEd and their actual implementations in online higher education (Kabudi et al., 2021). The discrepancy is caused by a separation of AI technology and the complex educational system (Xu & Ouyang, 2021). The review results also show a limited research work on examining the long-term effect and implication of applying AI to improve online instruction and learning. Therefore, AIEd needs to be designed and applied with the awareness that the AI technology is a part of a larger educational system, consisting of multiple components, e.g., learners, instructors, and information and resources (Riedl, 2019). To better examine the learning effects, AIEd empirical research should design more comprehensive assessment methods to incorporate various student features (e.g., student motivation, anxiety, higher-order thinking, behavioral patterns) in the AI model, and use multimodal learning analytics to collect and analyze data (e.g., process-oriented discourse data, physiological sensing data, eyetracking) (Ouyang & Jiao, 2021). In addition, the review indicates that most empirical research is conducted in a short period of time duration, therefore more empirical research is needed to enlarge the sample size and experiment duration in order to verify the effects of AI applications in online higher education. Overall, a deep understanding can be achieved by conducting more empirical research to examine the roles of AI in online higher education, educational and learning theories underpinned AIEd, and the actual effects of AI on online learning quality (Gartner, 2019; Law, 2019; Tegmark, 2017).
第三，从实践的角度来看，人工智能教育的发展需要更多的实证研究来探讨 在在线高等教育中的不同作用，人工智能如何与现有的教育和学习理论相联系，以及 技术的使用在多大程度上影响了在线学习的质量（Hwang 等，2020；Kabudi 等，2021；Ouyang & Jiao，2021）。正如研究人员在最近的文献综述中指出的，人工智能教育的潜力与在线高等教育中的实际应用之间存在差异（Kabudi et al.）造成这种差异的原因是人工智能技术与复杂的教育系统相分离（Xu & Ouyang, 2021）。综述结果还显示，关于应用人工智能改善在线教学的长期效果和影响的研究工作十分有限。因此，在设计和应用人工智能教育时，需要认识到人工智能技术是更大教育系统的一部分，由学习者、教师、信息和资源等多个部分组成（Riedl，2019）。为了更好地考察学习效果，人工智能教育实证研究应设计更全面的评估方法，将各种学生特征（如学生动机、焦虑、高阶思维、行为模式等）纳入人工智能模型，并使用多模态学习分析方法收集和分析数据（如面向过程的话语数据、生理传感数据、眼球跟踪等）（欧阳和焦，2021）。此外，综述表明，大多数实证研究都是在短时间内进行的，因此需要更多的实证研究来扩大样本量和实验时间，以验证人工智能在在线高等教育中的应用效果。总之，通过开展更多的实证研究，考察人工智能在在线高等教育中的作用、支撑人工智能教育的教育和学习理论，以及人工智能对在线学习质量的实际影响，可以加深对人工智能的理解（Gartner，2019；Law，2019；Tegmark，2017）。

6 Conclusions 6 结论

This systematic review provides an overview of empirical research on the applications of AI in online higher education. Specifically, this literature review examines the functions of empirical researches, the algorithms used in empirical researches and the effects and implications generated by empirical research. Although the research and practice of applications in online higher education are still in the
本系统性综述概述了在线高等教育中人工智能应用的实证研究。具体而言，本文献综述探讨了实证研究的功能、实证研究中使用的算法以及实证研究产生的效果和影响。尽管 在在线高等教育中的应用研究与实践仍处于起步阶段，但其对在线高等教育的影响已得到了广泛认可。
preliminary and exploratory stage, is proved to be positive to enhance online instruction and learning quality by offering accurate prediction, assessment and engaging students with online materials and environments (Yang et al., 2020; Zawacki-Richter et al., 2019). The innovative applications of AI in online higher education are conducive to reform instructional design and development methods, as well as advance the constructions of the intelligent, networked, personalized, and lifelong educational system (Arsovic & Stefanovic, 2020; Ouyang & Jiao, 2021; Yang et al., 2020).
在初步探索阶段， ，通过提供准确的预测、评估以及让学生参与在线材料和环境，被证明对提高在线教学和学习质量具有积极意义（Yang 等，2020；Zawacki-Richter 等，2019）。人工智能在在线高等教育中的创新应用有利于改革教学设计和开发方法，推进智能化、网络化、个性化和终身化教育体系的建设（Arsovic & Stefanovic, 2020; Ouyang & Jiao, 2021; Yang et al.）
This systematic review has several limitations, which lead to future research directions. First, the process of search query might not guarantee full completeness and absence of bias. Although we used the keyword list suggested by the previous review studies to search for the relevant articles, not all studies were included as diverse terms have been used to represent AI technologies. Second, the studies reviewed in this article were filtered from the seven prominent databases and the articles were limited to journal articles. For example, the recent conference proceedings were excluded, which may lead to the absence of the latest technical reports of AIEd in online higher education. Since AIEd is an interdisciplinary field where scholars come from different fields particularly computer science and education areas, studies might be published as conference papers that were not included. Therefore, future studies can adjust the screening criteria such that more relevant studies can be included. Third, the current study only provided a systematic overview of in online higher education, a formal meta-analysis would be beneficial to report the effect sizes of selected empirical research to gain a deeper understanding of the field.
本系统综述存在一些局限性，这也为今后的研究指明了方向。首先，搜索查询过程可能无法保证完全完整和无偏见。虽然我们使用了以往综述研究中建议的关键词列表来搜索相关文章，但由于代表人工智能技术的术语多种多样，因此并非所有研究都被包括在内。其次，本文回顾的研究是从七个著名数据库中筛选出来的，文章仅限于期刊论文。例如，最近的会议论文集被排除在外，这可能会导致缺少在线高等教育中人工智能教育的最新技术报告。由于人工智能教育是一个跨学科领域，学者来自不同的领域，尤其是计算机科学和教育领域，因此可能会有一些以会议论文形式发表的研究报告没有被收录。因此，今后的研究可以调整筛选标准，以便纳入更多相关研究。第三，目前的研究只是对 在线高等教育进行了系统的概述，正式的荟萃分析将有助于报告所选实证研究的效应大小，从而加深对该领域的理解。
Critical questions that need to be carefully considered include: How AI algorithms and models can be improved in online higher education? How AI systems or tools should be implemented to improve the instruction and learning practices in online higher education? How to conduct longitudinal empirical research in order to reveal authentic and long-term results of applying in online instruction and learning? This systematic review has provided initial implications for those questions, such as taking into consideration students' diverse characteristics, using advanced AI algorithms to achieve precisions, conducting longitudinal research to examine long-term effect of AI applications. Future work should continue on this research and practice trend. Overall, consistent with previous work (e.g., Deeva et al., 2021; Holmes et al., 2019; Hwang et al., 2020), AIEd applications in online higher education are expected to enable learners to reflect on learning and inform AI systems to adapt accordingly, improve prediction, recommendation and assessment accuracy, and facilitate learner agency, empowerment, and personalization in student-centered learning.
需要认真考虑的关键问题包括如何改进在线高等教育中的人工智能算法和模型？应如何实施人工智能系统或工具，以改进在线高等教育的教学和学习实践？如何开展纵向实证研究，以揭示在在线教学中应用 的真实和长期结果？本系统综述为这些问题提供了初步的启示，如考虑学生的不同特点、使用先进的人工智能算法实现精确性、开展纵向研究以考察人工智能应用的长期效果等。未来的工作应继续关注这一研究和实践趋势。总之，与之前的工作（如Deeva等人，2021；Holmes等人，2019；Hwang等人，2020）一致，在线高等教育中的人工智能教育应用有望使学习者能够反思学习，并告知人工智能系统进行相应调整，提高预测、推荐和评估的准确性，促进学习者在以学生为中心的学习中的能动性、赋权和个性化。

Appendix 附录

Tables 2, 3 and 4
表 2、表 3 和表 4
Acknowledgements This work is financially supported by the National Natural Science Foundation of China, No. 62177041.
致谢 本研究得到了国家自然科学基金（编号：62177041）的资助。

Declarations 声明

Conflict of interest There is no conflict of interest to declare.
利益冲突 没有利益冲突需要声明。

References 参考资料

* Reviewed articles
* 经审查的文章

  • Aguiar, E., Chawla, N. V., Brockman, J., Ambrose, G. A., & Goodrich, V. (2014). Engagement vs performance: using electronic portfolios to predict first semester engineering student retention. Journal of Learning Analytics, 1(3), 7-33. https://doi.org/10.18608/jla.2014.13.3
    Aguiar, E., Chawla, N. V., Brockman, J., Ambrose, G. A., & Goodrich, V. (2014).参与与表现：使用电子作品集预测工科学生第一学期的保留率》（Engagement vs Performance: Using electronic portfolios to predict first semester engineering student retention.Journal of Learning Analytics, 1(3), 7-33. https://doi.org/10.18608/jla.2014.13.3
  • Almeda, M. V., Zuech, J., Utz, C., Higgins, G., Reynolds, R., & Baker, R. S. (2018). Comparing the factors that predict completion and grades among for-credit and open/MOOC students in online learning. Online Learning, 22(1), 1-18. https://doi.org/10.24059/olj.v22i1.1060
    Almeda, M. V., Zuech, J., Utz, C., Higgins, G., Reynolds, R., & Baker, R. S. (2018)。比较预测在线学习中学分制学生和开放/MOOC学生完成学业和成绩的因素。Online Learning, 22(1), 1-18. https://doi.org/10.24059/olj.v22i1.1060
  • Aluthman, E. S. (2016). The effect of using automated essay evaluation on esl undergraduate students' writing skill. International Journal of English Linguistics, 6(5), 54. https://doi.org/10.5539/ijel. v6n5p54
    Aluthman, E. S. (2016).使用自动化论文评价对ESL本科生写作能力的影响。International Journal of English Linguistics, 6(5), 54. https://doi.org/10.5539/ijel. v6n5p54
Alyahyan, E., & Düştegör, D. (2020). Predicting academic success in higher education: literature review and best practices. In International Journal of Educational Technology in Higher Education (Vol. 17, Issue 1). Springer. https://doi.org/10.1186/s41239-020-0177-7
Alyahyan, E., & Düştegör, D. (2020)。预测高等教育中的学业成功：文献综述与最佳实践》。高等教育教育技术国际期刊》（第 17 卷第 1 期）。Springer. https://doi.org/10.1186/s41239-020-0177-7
Arsovic, B., & Stefanovic, N. (2020). E-learning based on the adaptive learning model:case study in Serbia. Sadhana-Academy Proceedings in Engineering Sciences, 45(1), 266. https://doi.org/10.1007/ s12046-020-01499-8.
Arsovic, B., & Stefanovic, N. (2020)。基于自适应学习模式的电子学习：塞尔维亚案例研究》。https://doi.org/10.1007/ s12046-020-01499-8.
  • Baneres, D., Rodriguez-Gonzalez, E., M., & Serra, M. (2019). An early feedback prediction system for learners at-risk within a first-year higher education course. IEEE Transactions on Learning Technologies, 12(2), 249-263. https://doi.org/10.1109/TLT.2019.2912167
    Baneres, D., Rodriguez-Gonzalez, E., M., & Serra, M. (2019)。针对高等教育一年级课程中高风险学习者的早期反馈预测系统。IEEE Transactions on Learning Technologies, 12(2), 249-263. https://doi.org/10.1109/TLT.2019.2912167
  • Benhamdi, S., Babouri, A., & Chiky, R. (2017). Personalized recommender system for e-Learning environment. Education and Information Technologies, 22(4), 1455-1477. https://doi.org/10.1007/ s10639-016-9504-y
    Benhamdi, S., Babouri, A., & Chiky, R. (2017)。电子学习环境的个性化推荐系统。教育与信息技术》，22（4），1455-1477。https://doi.org/10.1007/ s10639-016-9504-y
  • Bousbahi, F., & Chorfi, H. (2015). MOOC-Rec: A case based recommender system for MOOCs. Procedia - Social and Behavioral Sciences, 195, 1813-1822. https://doi.org/10.1016/j.sbspro.2015.06.395
    Bousbahi, F., & Chorfi, H. (2015)。MOOC-Rec：基于案例的 MOOCs 推荐系统。Procedia - Social and Behavioral Sciences, 195, 1813-1822. https://doi.org/10.1016/j.sbspro.2015.06.395
Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013). Studying learning in the worldwide classroom: Research into edX's first MOOC. Research & Practice in Assessment, 8, 13-25. Retrieved from https://files.eric.ed.gov/fulltext/EJ1062850.pdf
Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013).研究全球课堂中的学习：对 edX 首个 MOOC 的研究。Research & Practice in Assessment, 8, 13-25.取自 https://files.eric.ed.gov/fulltext/EJ1062850.pdf
  • Burgos, C., Campanario, M. L., de la Peña, D., Lara, J. A., Lizcano, D., & Martínez, M. A. (2018). Data mining for modeling students' performance: A tutoring action plan to prevent academic dropout. Computers and Electrical Engineering, 66, 541-556. https://doi.org/10.1016/j.compeleceng. 2017.03 .005
    Burgos, C., Campanario, M. L., de la Peña, D., Lara, J. A., Lizcano, D., & Martínez, M. A. (2018)。学生成绩建模的数据挖掘：防止辍学的辅导行动计划。计算机与电气工程》，66，541-556。https://doi.org/10.1016/j.compeleceng.2017.03 .005
  • Cárdenas-Cobo, J., Puris, A., Novoa-Hernández, P., Galindo, J. A., & Benavides, D. (2020). Recommender systems and scratch: An integrated approach for enhancing computer programming learning. IEEE Transactions on Learning Technologies, 13(2), 387-403. https://doi.org/10.1109/TLT. 2019.2901457
    Cárdenas-Cobo, J., Puris, A., Novoa-Hernández, P., Galindo, J. A., & Benavides, D. (2020)。推荐系统与划痕：增强计算机编程学习的综合方法。IEEE Transactions on Learning Technologies, 13(2), 387-403. https://doi.org/10.1109/TLT.2019.2901457
Castañeda, L., & Selwyn, N. (2018). More than tools? Making sense of the ongoing digitizations of higher education. International Journal of Educational Technology in Higher Education, 15(22), https://doi.org/10.1186/s41239-018-0109-y
Castañeda, L., & Selwyn, N. (2018)。不仅仅是工具？理解高等教育正在进行的数字化。高等教育教育技术国际期刊》，15（22），https://doi.org/10.1186/s41239-018-0109-y。
Chassignol, M., Khoroshavin, A., Klimova, A., & Bilyatdinova, A. (2018). Artificial intelligence trends in education: A narrative overview. 7th International Young Scientist Conference on Computational Science. Procedia Computer Science, 136, 16-24
Chassignol, M., Khoroshavin, A., Klimova, A., & Bilyatdinova, A. (2018).教育领域的人工智能趋势：叙述性概述。第七届国际青年科学家计算科学大会。Procedia Computer Science, 136, 16-24
Chang, T. Y., & Ke, Y. R. (2013). A personalized e-course composition based on a genetic algorithm with forcing legality in an adaptive learning system. Journal of Network and Computer Applications, 36(1), 533-542. https://doi.org/10.1016/j.jnca.2012.04.002
Chang, T. Y., & Ke, Y. R. (2013).自适应学习系统中基于强迫合法性遗传算法的个性化电子课程组成。网络与计算机应用期刊》，36（1），533-542。https://doi.org/10.1016/j.jnca.2012.04.002。
Chen, C. M. (2008). Intelligent web-based learning system with personalized learning path guidance. Computers & Education, 5l(2), 787-814. https://doi.org/10.1016/j.compedu.2007.08.004
Chen, C. M. (2008).具有个性化学习路径指导的智能网络学习系统。计算机与教育，5l（2），787-814。https://doi.org/10.1016/j.compedu.2007.08.004。
  • Chen, W., Niu, Z., Zhao, X., & Li, Y. (2014). A hybrid recommendation algorithm adapted in e-learning environments. World Wide Web, 17(2), 271-284. https://doi.org/10.1007/s11280-012-0187-z
    Chen, W., Niu, Z., Zhao, X., & Li, Y. (2014)。适用于电子学习环境的混合推荐算法。World Wide Web, 17(2), 271-284. https://doi.org/10.1007/s11280-012-0187-z
Chen, X., Xie, H., & Hwang, G. J. (2020a). A multi-perspective study on artificial intelligence in education: grants, conferences, journals, software tools, institutions, and researchers. Computers and Education: Artificial Intelligence, 1, 100005. https://doi.org/10.1016/j.caeai.2020.100005 
Chen, X., Xie, H., Zou, D., & Hwang, G. J. (2020b). Application and theory gaps during the rise of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1(July), 100002. https://doi.org/10.1016/j.caeai.2020.100002 
  • Christudas, B. C. L., Kirubakaran, E., & Thangaiah, P. R. J. (2018). An evolutionary approach for personalization of content delivery in e-learning systems based on learner behavior forcing compatibility of learning materials. Telematics and Informatics, 35(3), 520-533. https://doi.org/10. j.tele. 2017.02 .004 
  • Costa, E. B., Fonseca, B., Santana, M. A., de Araújo, F. F., & Rego, J. (2017). Evaluating the effectiveness of educational data mining techniques for early prediction of students' academic failure in introductory programming courses. Computers in Human Behavior, 73, 247-256. https://doi. org/10.1016/j.chb.2017.01.047 
Deeva, G., Bogdanova, D., Serral, E., Snoeck, M., & De Weerdt, J. (2021). A review of automated feedback systems for learners: Classification framework, challenges and opportunities. Computers & Education, 162, 104094. https://doi.org/10.1016/j.compedu.2020.104094 
Du Boulay, B. (2000). Can we learn from ITSs? In International conference on intelligent tutoring systems (pp. 9-17). Springer. https://link.springer.com/chapter/10.1007/3-540-45108-0_3 
  • Dwivedi, P., & Bharadwaj, K. K. (2015). E-Learning recommender system for a group of learners based on the unified learner profile approach. Expert Systems, 32(2), 264-276. https://doi.org/10. 1111/exsy. 12061 
Gardner, L., Sheridan, D., & White, D. (2002). A web-based learning and assessment system to support flexible education. Journal of Computer Assisted Learning, 18, 125e136. https://doi.org/10.  
Gartner (2019). Hype cycle for emerging technologies, 2019. Gartner. Retrieved on 2021/1/1 https:// www.gartner.com/en/documents/3956015/hype-cycle-for-emerging-technologies-2019 
George, G., & Lal, A. M. (2019). Review of ontology-based recommender systems in e-learning. Computers & Education, 142(July), 103642. https://doi.org/10.1016/j.compedu.2019.103642 
Giannakos, M. N., Sharma, K., Pappas, I. O., Kostakos, V., & Velloso, E. (2019). Multimodal data as a means to understand the learning experience. International Journal of Information Management, 48, 108-119. https://doi.org/10.1016/j.ijinfomgt.2019.02.003 
Godwin, A., & Kirn, A. (2020). Identity-based motivation: Connections between first-year students' engineering role identities and future-time perspectives. Journal of Engineering Education, 109(3), 362-383. https://doi.org/10.1002/jee. 20324 
Graneheim, U. H., & Lundman, B. (2004). Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105e112. https://doi.org/10.1016/j.nedt.2003.10.001 
Guan, C., Mou, J., & Jiang, Z. (2020). Artificial intelligence innovation in education: A twenty-year data-driven historical analysis. International Journal of Innovation Studies, 4(4), 134-147. https://doi.org/10.1016/j.ijis.2020.09.001 
Harasim, L. (2000). Shift happens: Online education as a new paradigm in learning. The Internet and higher education, 3(1-2), 41-61. https://doi.org/10.1016/S1096-7516(00)00032-4 
  • Helal, S., Li, J., Liu, L., Ebrahimie, E., Dawson, S., Murray, D. J., & Long, Q. (2018). Predicting academic performance by considering student heterogeneity. Knowledge-Based Systems, 161(July), 134-146. https://doi.org/10.1016/j.knosys.2018.07.042 
Henly, D. C. (2003). Use of web-based formative assessment to support student learning in a metabolism/nutrition unit. European Journal of Dental Education, 7(3), 116e122. https://doi.org/10.  
  • Hew, K. F., Hu, X., Qiao, C., & Tang, Y. (2020). What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach. Computers Education, 145, 103724. https://doi.org/10.1016/j.compedu.2019.103724 
Hinojo-Lucena, F. J., Aznar-Díaz, I., Cáceres-Reche, M. P., & Romero-Rodríguez, J. M. (2019). Artificial intelligence in higher education: A bibliometric study on its impact in the scientific literature. Education Sciences, 9(1), 51. https://doi.org/10.3390/educsci9010051 
Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cognition: Toward a new foundation for human-computer interaction research. ACM Transactions on Computer-Human Interaction (TOCHI), 7(2). https://doi.org/10.1145/353485.353487 
Holmberg, B. (2005). Theory and practice of distance education. Routledge 
Holmes, W., Bialik, M., & Fadel, C. (2019). Artifficial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign 
  • Hooshyar, D., Ahmad, R. B., Yousefi, M., Fathi, M., Horng, S. J., & Lim, H. (2016). Applying an online game-based formative assessment in a flowchart-based intelligent tutoring system for improving problem-solving skills. Computers & Education, 94, 18-36. https://doi.org/10.1016/j.compedu. 2015.10 .013 
  • Howard, E., Meehan, M., & Parnell, A. (2018). Contrasting prediction methods for early warning systems at undergraduate level. The Internet and Higher Education, 37, 66-75. https://doi.org/10. 1016/j.iheduc. 2018.02.001 
Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288. https://doi.org/10.1177/1049732305276687 
Hu, Y. H. (2021). Effects and acceptance of precision education in an AI-supported smart learning environment. Education and Information Technologies. https://doi.org/10.1007/s10639-021-10664-3 
  • Hu, Y., Lo, C., & Shih, S. (2014). Developing early warning systems to predict students' online learning performance. Computers in Human Behavior, 36, 469-478. https://doi.org/10.1016/j.chb.2014. 04.002 
  • Huang, A. Y. Q., Lu, O. H. T., Huang, J. C. H., Yin, C. J., & Yang, S. J. H. (2020). Predicting students' academic performance by using educational big data and learning analytics: evaluation of classification methods and learning logs. Interactive Learning Environments, 28(2), 206-230. https://doi.org/  
Hwang, G. J., & Tu, Y. F. (2021). Roles and research trends of artificial intelligence in mathematics education: A bibliometric mapping analysis and systematic review. Mathematics. https://doi.org/10. 3390/math 9060584 
Hwang, G. J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, 100001. https://doi.org/10.1016/j.caeai.2020.100001 
  • Ijaz, K., Bogdanovych, A., & Trescak, T. (2017). Virtual worlds vs books and videos in history education. Interactive Learning Environments, 25(7), 904-929. https://doi.org/10.1080/10494820.2016. 1225099 
  • Jayaprakash, S. M., Moody, E. W., Eitel, J. M., Regan, J. R., & Baron, J. D. (2014). Early alert of academically at-risk students: An open source analytics initiative. Journal of Learning Analytics, 1, . https://doi.org/10.18608/jla.2014.11.3 
Kabudi, T., Pappas, I., & Olsen, D. H. (2021). AI-enabled adaptive learning systems: A systematic mapping of the literature. Computers and Education: Artificial Intelligence, 2, 100017. https://doi.org/ . caeai. 2021.100017 
Kim, J. N., & Grunig, J. E. (2011). Problem solving and communicative action: A situational theory of problem solving. Journal of Communication, 61(1), 120-149. https://doi.org/10.1111/j.1460-2466. 2010.01529.x 
  • Koć-Januchta, M. M., Schönborn, K. J., Tibell, L. A. E., Chaudhri, V. K., & Heller, H. C. (2020). Engaging with biology by asking questions: Investigating students' interaction and learning with an artificial intelligence-enriched textbook. Journal of Educational Computing Research, 58(6), 11901224. https://doi.org/10.1177/0735633120921581 
Law, N. W. Y. (2019). Human development and augmented intelligence. In The 20th international conference on artificial intelligence in education (AIED 2019). Springer. Retrieved on 2021/1/1 from https://www.sciencedirect.com/science/refhub/S2666-920X(21)00014-undefined/sref31 
  • Li, J., Chang, Y., Chu, C., & Tsai, C. (2012). Expert systems with applications a self-adjusting e-course generation process for personalized learning. Expert Systems With Applications, 39(3), 3223-3232. https://doi.org/10.1016/j.eswa.2011.09.009 
Liu, M., Kang, J., Zou, W., Lee, H., Pan, Z., & Corliss, S. (2017). Using data to understand how to better design adaptive learning. Technology Knowledge and Learning, 22(3), 271-298. https://doi.org/10.  
Liu, S., Guo, D., Sun, J., Yu, J., & Zhou, D. (2020). MapOnLearn: The use of maps in online learning systems for education sustainability. Sustainability, 12(17), 7018. https://doi.org/10.3390/su121 77018 
Liz-Domínguez, M., Caeiro-Rodríguez, M., Llamas-Nistal, M., & Mikic-Fonte, F. A. (2019). Systematic literature review of predictive analysis tools in higher education. Applied Sciences (Switzerland), 9(24). MDPI AG. https://doi.org/10.3390/app9245569 
Lykourentzou, I., Giannoukos, I., Nikolopoulos, V., Mpardis, G., & Loumos, V. (2009). Dropout prediction in e-learning courses through the combination of machine learning techniques. Computers & Education, 53(3), 950-965. https://doi.org/10.1016/j.compedu.2009.05.010 
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097.t001 
Moreno-Marcos, P. M., Muñoz-Merino, P. J., Alario-Hoyos, C., Estévez-Ayres, I., & Delgado Kloos, C. (2018). Analysing the predictive power for anticipating assignment grades in a massive open online course. Behaviour & Information Technology, 37(10-11), 1021-1036. https://doi.org/10.1080/ 0144929X.2018.1458904 
Moreno-Marcos, P. M., Alario-Hoyos, C., Munoz-Merino, P. J., & Kloos, C. D. (2019). Prediction in MOOCs: A Review and Future Research Directions. IEEE Transactions on Learning Technologies, 12(3), 384-401. https://doi.org/10.1109/TLT.2018.2856808 
Moseley, L. G., & Mead, D. M. (2008). Predicting who will drop out of nursing courses: a machine learning exercise. Nurse Education Today, 28(4), 469-475. https://doi.org/10.1016/j.nedt.2007.07.012 
  • Mubarak, A. A., Cao, H., & Zhang, W. (2020). Prediction of students' early dropout based on their interaction logs in online learning environment. Interactive Learning Environments. https://doi.org/
    Mubarak, A. A., Cao, H., & Zhang, W. (2020)。基于在线学习环境中学生互动日志的学生提前辍学预测。互动学习环境。https://doi.org/
Neuendorf, K. A., & Kumar, A. (2015). Content analysis. The international Encyclopedia of Political Communication, 1-10. https://doi.org/10.1002/9781118541555.wbiepc065
Neuendorf, K. A., & Kumar, A. (2015).The international Encyclopedia of Political Communication, 1-10.The international Encyclopedia of Political Communication, 1-10. https://doi.org/10.1002/9781118541555.wbiepc065
Ouyang, F. & Jiao, P. (2021). Artificial Intelligence in Education: The Three Paradigms. Computers & Education: Artificial Intelligence, 100020. https://doi.org/10.1016/j.caeai.2021.100020
Ouyang, F. & Jiao, P. (2021)。人工智能在教育中的应用：三大范式》。计算机与教育：人工智能》，100020。https://doi.org/10.1016/j.caeai.2021.100020。
Rico-Juan, J. R., Gallego, A. J., & Calvo-Zaragoza, J. (2019). Automatic detection of inconsistencies between numerical scores and textual feedback in peer-assessment processes with machine learning. Computers & Education, 140, 103609. https://doi.org/10.1016/j.compedu.2019.103609
Rico-Juan, J. R., Gallego, A. J., & Calvo-Zaragoza, J. (2019)。利用机器学习自动检测互评过程中数字分数与文本反馈之间的不一致性。Computers & Education, 140, 103609. https://doi.org/10.1016/j.compedu.2019.103609
Riedl, M. O. (2019). Human-centered artificial intelligence and machine learning. Human Behavior and Emerging Technologies, 1(1), 33-36. https://doi.org/10.1002/hbe2.117
Riedl, M. O. (2019).以人为本的人工智能和机器学习》。人类行为与新兴技术》，1（1），33-36。https://doi.org/10.1002/hbe2.117。
Rowe, M. (2019). Shaping our algorithms before they shape us. Artificial Intelligence and Inclusive Education (pp. 151-163). Springer. https://doi.org/10.1007/978-981-13-8161-4_9
Rowe, M. (2019).在算法塑造我们之前先塑造算法。人工智能与全纳教育》（第 151-163 页）。Springer. https://doi.org/10.1007/978-981-13-8161-4_9
Romero, C., Espejo, P. G., Zafra, A., Romero, J. R., & Ventura, S. (2013). Web usage mining for predicting final marks of students that use Moodle courses. Computer Applications in Engineering Education, 21(1), 135-146. https://doi.org/10.1002/cae. 20456
Romero, C., Espejo, P. G., Zafra, A., Romero, J. R., & Ventura, S. (2013)。使用 Moodle 课程预测学生期末分数的网络使用挖掘。工程教育中的计算机应用》，21（1），135-146。https://doi.org/10.1002/cae。20456
  • Romero, C., López, M. I., Luna, J. M., & Ventura, S. (2013). Predicting students' final performance from participation in on-line discussion forums. Computers & Education, 68, 458-472. https://doi. org/10.1016/j.compedu.2013.06.009
    Romero, C., López, M. I., Luna, J. M., & Ventura, S. (2013)。通过参与在线论坛预测学生期末成绩》。https://doi. org/10.1016/j.compedu.2013.06.009
Selwyn, N. (2016). Is technology good for education? Polity Press. Retrieved on 2021/1/10 from http:// au.wiley.com/WileyCDA/WileyTitle/productCd-0745696465.html
Selwyn, N. (2016).技术对教育有益吗？Polity Press.2021/1/10 检索自 http:// au.wiley.com/WileyCDA/WileyTitle/productCd-0745696465.html
Shahiri, A. M., Husain, W., & Rashid, N. A. (2015). A review on predicting student's performance using data mining techniques. Procedia Computer Science, 72, 414-422. https://doi.org/10.1016/j.procs. 2015.12 .157
Shahiri, A. M., Husain, W., & Rashid, N. A. (2015)。使用数据挖掘技术预测学生成绩的综述。Procedia Computer Science, 72, 414-422. https://doi.org/10.1016/j.procs.2015.12 .157
Simpson, O. (2018). Supporting students in online, open and distance learning (1st ed.). Routledge
Simpson, O. (2018).在线、开放和远程学习中的学生支持（第 1 版）。Routledge
  • Sukhbaatar, O., Usagawa, T., & Choimaa, L. (2019). An artificial neural network based early prediction of failure-prone students in blended learning course. International Journal of Emerging Technologies in Learning, 14(19), 77-92. https://doi.org/10.3991/ijet.v14i19.10366
    Sukhbaatar, O., Usagawa, T., & Choimaa, L. (2019)。基于人工神经网络的混合式学习课程中易失败学生的早期预测。国际学习新兴技术期刊》，14（19），77-92。https://doi.org/10.3991/ijet.v14i19.10366。
Tang, K. Y., Chang, C. Y., & Hwang, G. J. (2021). Trends in artificial intelligence supported e-learning: A systematic review and co-citation network analysis (1998-2019). Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.1875001
Tang, K. Y., Chang, C. Y., & Hwang, G. J. (2021).人工智能支持的电子学习趋势：人工智能支持的电子学习趋势：系统综述与联合引用网络分析（1998-2019 年）。交互式学习环境。https://doi.org/10.1080/10494820.2021.1875001
Tegmark, M. (2017). Life 3.0: Being human in the age of artificial intelligence. (Knopf)
Tegmark, M. (2017).生命 3.0：人工智能时代的人类。(诺普夫出版社）
Tomasevic, N., Gvozdenovic, N., & Vranes, S. (2020). An overview and comparison of supervised data mining techniques for student exam performance prediction. Computers & Education, 143, 103676. https://doi.org/10.1016/j.compedu.2019.103676
Tomasevic, N., Gvozdenovic, N., & Vranes, S. (2020)。用于学生考试成绩预测的监督数据挖掘技术概述与比较》。计算机与教育》，143，103676。https://doi.org/10.1016/j.compedu.2019.103676。
Tyler-Smith, K. (2006). Early attrition among first time eLearners: A review of factors that contribute to drop-out, withdrawal and non-completion rates of adult learners undertaking eLearning programmes. Journal of Online Learning and Teaching, 2(2), 73-85. Retrieved on 2021/1/11 from https://jolt.merlot.org/documents/Vol2_No2_TylerSmith_000.pdf
Tyler-Smith, K. (2006)。首次电子学习者的早期流失：对导致成人学习者辍学率、退出率和未完成学业率的因素的回顾。在线学习与教学期刊》，2（2），73-85。2021/1/11 从 https://jolt.merlot.org/documents/Vol2_No2_TylerSmith_000.pdf 检索
Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. Harvard University Press
Vygotsky, L. (1978).社会中的心灵：The development of higher psychological processes.哈佛大学出版社
  • Wakelam, E., Jefferies, A., Davey, N., & Sun, Y. (2020). The potential for student performance prediction in small cohorts with minimal available attributes. British Journal of Educational Technology, 51(2), 347-370. https://doi.org/10.1111/bjet. 12836
    Wakelam, E., Jefferies, A., Davey, N., & Sun, Y. (2020)。用最少的可用属性预测小规模队列中学生成绩的潜力》。英国教育技术期刊》，51（2），347-370。https://doi.org/10.1111/bjet。12836
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th international conference on evaluation and assessment in software engineering (pp. 1-10). https://doi.org/10.1145/2601248.2601268
Wohlin, C. (2014)。系统文献研究中的滚雪球指南及在软件工程中的复制。第 18 届软件工程评估与评价国际会议论文集》（第 1-10 页）。https://doi.org/10.1145/2601248.2601268。
Xie, H., Chu, H. C., Hwang, G. J., & Wang, C. C. (2019). Trends and development in technologyenhanced adaptive/personalized learning: A systematic review of journal publications from 2007 to 2017. Computers & Education, 140, 103599. https://doi.org/10.1016/j.compedu.2019.103599
Xie, H., Chu, H. C., Hwang, G. J., & Wang, C. C. (2019).技术增强型自适应/个性化学习的趋势与发展：2007-2017年期刊论文的系统回顾。Computers & Education, 140, 103599. https://doi.org/10.1016/j.compedu.2019.103599
  • Xing, W., Chen, X., Stein, J., & Marcinkowski, M. (2016). Computers in human behavior temporal predication of dropouts in MOOCs: Reaching the low hanging fruit through stacking generalization. Computers in Human Behavior, 58, 119-129. https://doi.org/10.1016/j.chb.2015.12.007
    Xing, W., Chen, X., Stein, J., & Marcinkowski, M. (2016)。人类行为中的计算机对 MOOCs 辍学的时间预测：通过堆叠泛化达到低悬果。人类行为中的计算机》，58, 119-129. https://doi.org/10.1016/j.chb.2015.12.007
Xu, W. & Ouyang, F. (2021). A systematic review of AI role in the educational system based on a proposed conceptual framework. Education and Information Technologies. https://doi.org/10.1007/ s10639-021-10774-y
Xu, W. & Ouyang, F. (2021).基于拟议概念框架的人工智能在教育系统中的作用系统综述.https://doi.org/10.1007/ s10639-021-10774-y
  • Xu, X., Wang, J., Peng, H., & Wu, R. (2019). Prediction of academic performance associated with internet usage behaviors using machine learning algorithms. Computers in Human Behavior, 98(April), 166-173. https://doi.org/10.1016/j.chb.2019.04.015
    Xu, X., Wang, J., Peng, H., & Wu, R. (2019).利用机器学习算法预测与互联网使用行为相关的学习成绩。Computers in Human Behavior, 98(April), 166-173. https://doi.org/10.1016/j.chb.2019.04.015
  • Yang, T., Brinton, C. G., & Joe-wong, C. (2017). Behavior-based grade prediction for MOOCs via time series neural networks. IEEE Journal of Selected Topics in Signal Processing, 11(5), 716-728. https://doi.org/10.1109/JSTSP.2017.2700227
    Yang, T., Brinton, C. G., & Joe-wong, C. (2017)。通过时间序列神经网络进行基于行为的 MOOC 成绩预测。IEEE Journal of Selected Topics in Signal Processing, 11(5), 716-728. https://doi.org/10.1109/JSTSP.2017.2700227
Yang, Y. T. C., Gamble, J. H., Hung, Y. W., & Lin, T. Y. (2014). An online adaptive learning environment for critical-thinking-infused English literacy instruction. British Journal of Educational Technology, 45(4), 723-747. https://doi.org/10.1111/bjet. 12080
Yang, Y. T. C., Gamble, J. H., Hung, Y. W., & Lin, T. Y. (2014).批判性思维注入英语读写教学的在线自适应学习环境。British Journal of Educational Technology, 45(4), 723-747. https://doi.org/10.1111/bjet.12080
Yang, C., Huan, S., & Yang, Y. (2020). A practical teaching mode for collegessupported by Artificial Intelligence. International Journal of Emerging Technologies in Learning, 15(17), 195-206. https:// doi.org/10.3991/ijet.v15i17.16737 
  • Yoo, J., & Kim, J. (2014). Project performance? Investigating the roles of linguistic features and participation patterns. International Journal of Artificial Intelligence in Education, 24, 8-32. https://doi. org/10.1007/s40593-013-0010-8 
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education - where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/ s41239-019-0171-0 
Zhai, X., Chu, X., Chai, C. S., Siu, M., Jong, Y., Istenic, A. ... Li, Y. (2021). A Review of Artificial Intelligence (AI) in Education from 2010 to 2020. 2021. Complexity, 8812542. https://doi.org/10.1155/  
  • Zohair, L. M. (2019). Prediction of student's performance by modelling small dataset size. International Journal of Educational Technology in Higher Education, 16(1). https://doi.org/10.1186/ s41239-019-0160-3 
Zupic, I., & Čater, T. (2015). Bibliometric methods in management and organization. Organizational Research Methods, 18(3), 429-472. https://doi.org/10.1177/1094428114562629 
Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. 

Authors and Affiliations 

Fan Ouyang Luyi Zheng Pengcheng Jiao  

Fan Ouyang 

fanouyang zju.edu.cn 
1 College of Education, Zhejiang University, Hangzhou, Zhejiang 310000, China 
2 Ocean College, Zhejiang University, Zhoushan, Zhejiang 316021, China 
  1. Extended author information available on the last page of the article