Exploring the Interaction Kinesics of a Soft Social Robot 探索软社交机器人的交互运动学
Ulrich Farhadi , Troels Aske Klausen , Ulrich Farhadi , Troels Aske Klausen 、Jonas Jørgensen , and Evgenios 乔纳斯-约根森 和叶夫根尼欧斯Vlachos 弗拉乔斯 Faculty of Engineering (contributed equally to this work) 工程学院（对本工作有同等贡献） Center for Soft Robotics, SDU Biorobotics, Mærsk Mc-Kinney Møller Institute 软机器人中心、SDU 生物机器人学、Mærsk Mc-Kinney Møller 研究所 Mærsk Mc-Kinney Møller Institute and University Library of Southern Denmark University of Southern Denmark (SDU), Odense, Denmark. 南丹麦大学（SDU），丹麦欧登塞。+ Address all correspondence to evvl@mmmi.sdu.dk + 将所有信件发送至 evvl@mmmi.sdu.dk
Abstract 摘要
In this pilot study, we explore the kinesics of a non-humanoid pneumatically actuated soft robot developed for social human-robot interaction (HRI). The robot uses motion and gesture to communicate: it can tilt; expand; and perform movements reminiscent of breathing. To explore the robot's kinesics, we hand-coded 8 movement presets intended to represent a specific action, internal state/emotion, or pattern of motion: greeting; avoid; breathless; joyful; alarming; jellyfish; frighten; sigh. We then conducted an online survey where participants chose words to describe their perception of each preset behavior. Semantic analysis of word choices indicates that the intended meaning was conveyed to users for most of the presets. Analysis of the general comments with text mining techniques showed that the robot was perceived as resembling an animal-like sea creature or a human body part (lungs, belly, or heart) and as communicating distinct emotional states. The results indicate that it is possible to communicate with a user solely through soft robotic movement, and suggest that soft robotics has applications within the design of affective interfaces and social robotics. 在这项试验性研究中，我们探索了一个非人类气动软体机器人的运动学，该机器人是为社交人机交互（HRI）而开发的。该机器人利用动作和手势进行交流：它可以倾斜、伸缩，还能做出类似呼吸的动作。为了探索机器人的运动学，我们手工编写了 8 种运动预设，旨在代表特定的动作、内部状态/情绪或运动模式：问候；回避；喘不过气来；高兴；惊慌；水母；惊吓；叹息。然后，我们进行了一项在线调查，让参与者 选择词语来描述他们对每种预设行为的感知。对选词的语义分析表明，大多数预设行为都向用户传达了预期的含义。利用文本挖掘技术对一般性评论进行的分析表明，机器人被认为类似于动物般的海洋生物或人类的身体部位（肺、腹部或心脏），并传达了不同的情感状态。结果表明，仅通过软体机器人的动作与用户交流是可能的，并表明软体机器人技术在情感界面和社交机器人设计中具有应用前景。
Nonverbal communication (NVC) plays a significant role in human-human interaction and research has shown that up to of human communication can be nonverbal 1. Humans unintentionally emit nonverbal cues related to their emotional state that help others understand the their actions and verbal utterances. Kinesics is defined as nonverbal communication through facial expressions, gestures, body movements and positioning 2 . Kinesics are intuitively and routinely used in human interaction and compared to other forms of NVC, kinesics is highly informative and even on level with verbal communication [3, and may convey 非语言沟通（NVC）在人际交往中发挥着重要作用，研究表明，人类沟通中高达 的沟通可能是非语言的1。人类会无意中发出与自己情绪状态有关的非语言暗示，帮助他人理解自己的行为和言语。动作学被定义为通过面部表情、手势、身体动作和姿势进行的非语言交流 2 。与其他形式的非语言沟通相比，动觉具有很强的信息量，甚至可以与语言沟通相提并论[3]。
rich information pertaining to both context and social interaction dynamics 4 . As humans have the innate psychological trait of anthropomorphism, robotic behaviors can be made more understandable by utilizing kinesics from human communication even in cases of non-humanoid social human-robot interaction (HRI). Kinesics-based robotics research has focused on arm gestures, body and head movements, eye gaze, and facial expressions 5]. Body kinesics encompass both static postures and dynamic movement. Body posture has been shown to define the steps and order in human interaction [6], while head movements are used to communicate a referent in narration through e.g., nodding or indexing 7. 8]. 与上下文和社会互动动态有关的丰富信息4 。由于人类与生俱来就有拟人化的心理特征，因此即使在非人类社会性人机交互（HRI）的情况下，也可以利用人类交流中的运动学来使机器人行为更容易理解。基于运动学的机器人研究主要集中在手臂手势、身体和头部运动、眼睛注视和面部表情 5]。身体运动学包括静态姿势和动态运动。身体姿态已被证明可确定人机交互的步骤和顺序[6]，而头部动作则可通过点头或示意等方式在叙述中传达指代内容[7]。
In this pilot study, we explore a soft silicone-based robot's potential for kinesics. We developed a custom-made pneumatically actuated soft robot specifically for social HRI. Research on soft robotics usually consider the tactile dimension of HRI, which is critical considering the increasing need for assistive robots 9]. However, prior work has shown that motion and shape-change in soft robots can be interpreted as socially communicative . Hence, our robot solely relies on bodily motion and posture to communicate with a human interaction partner: it is able to tilt and expand its upper and lower parts and can perform movements reminiscent of breathing. 在这项试验研究中，我们探索了硅基软体机器人在运动学方面的潜力。我们开发了一个定制的气动软体机器人，专门用于社交人机交互。有关软体机器人的研究通常会考虑人机交互的触觉维度，考虑到对辅助机器人日益增长的需求，这一点至关重要 9]。然而，先前的研究表明，软体机器人的运动和形状变化可以被解释为具有社会交流功能 。因此，我们的机器人仅依靠身体运动和姿势来与人类互动伙伴交流：它能够倾斜和伸展上下部分，并能做出类似呼吸的动作。
2 Materials and Methods 2 材料与方法
2.1 Robot 2.1 机器人
We chose an abstract zoomorphic design, as we did not want the robot to resemble nor a human neither a familiar pet, yet it should be perceived as endowed with the qualities of a living organism. The morphology has a rounded, organic shape that somewhat resembles a jellyfish (Fig. 11). The bottom part of the robot is split into three individually controllable separate air chambers. They allow the robot to tilt towards every direction within a full circle of 360 degrees, and also enable upward movement if inflated simultaneously. The top part houses a single chamber that when inflated makes the robot expand upwards, and increase in width. Small indentations in the silicone material were added to the top's surface to give it a textured skin-like appearance. The robot is 12 cm in diameter, and has a height of 6.2 cm . It was manufactured from Ecoflex 00-30 silicone through a multi-step casting process using 3D printed molds 14 . An Arduino UNO R3 with a custom 8 channel motor shield was used to control 4 electrical pumps and 4 solenoid valves for actuation using pneumatic supply tubing with ID/OD. 我们选择了一种抽象的变形设计，因为我们不想让机器人既不像人，也不像我们熟悉的宠物，但又要让人觉得它具有生物体的特质。机器人的形态呈圆形，有点像水母（图 11）。机器人的底部分为三个可单独控制的独立气室。它们可以让机器人在 360 度的圆周范围内向各个方向倾斜，如果同时充气，还能实现向上运动。顶部有一个气室，充气后机器人会向上膨胀，宽度也会增加。顶部表面的硅胶材料上还增加了小凹痕，使其看起来像皮肤一样有纹理。机器人直径为 12 厘米，高度为 6.2 厘米。它由 Ecoflex 00-30 硅胶制成，使用 3D 打印模具 14 通过多步铸造工艺制成。Arduino UNO R3 配有一个定制的 8 通道电机屏蔽，用于控制 4 个电动泵和 4 个电磁阀，使用内径/外径 的气动供气管道进行驱动。
2.2 Movement Presets 2.2 运动预设
To explore the robot's kinesics, 8 expressive movement presets were hand-coded. The first 3 were adapted from existing socially communicative movements described in the literature. The first two were based on the "approach" and the 为了探索机器人的运动学，我们手工编码了 8 个富有表现力的预设动作。前三个是根据文献中描述的现有社会交流动作改编的。前两个是基于 "接近 "和
Fig. 1. CAD rendering of the soft robot morphology (left), and screenshots from Video 1 showing the physical robot (right). 图 1.软体机器人形态的 CAD 渲染图（左）和显示实体机器人的视频 1 截图（右）。
"avoid" gesture used by the abstract robotic object the "Greeting Machine" [15]. In prior work these were found to respectively be perceived as signaling that the robot was and was not available for social interaction (i.e., a positive and negative opening cue). The third preset was based on a study finding that observing breath-holding in another person induces the effect of "breathless" (the feeling of not being able to breathe enough) in the observer 16. The remaining 5 presets (joyful, alarming, jellyfish, frighten, sigh) were exploratory non-validated movements that we designed ourselves. An overview of the presets and hyperlinks for each video stimulus are provided in Table 1 而 "打招呼机器"（Greeting Machine）[15] 这一抽象机器人对象则使用了 "回避 "手势。在之前的研究中，这些手势分别被认为是机器人可以和不可以进行社交互动的信号（即积极和消极的开放线索）。第三个预设是基于一项研究发现，观察他人憋气会诱发观察者产生 "喘不过气"（呼吸不够的感觉）的效果16。其余 5 个预设动作（欢乐、惊恐、水母、惊吓、叹息）是我们自己设计的未经验证的探索性动作。表 1 提供了每个视频刺激的预设和超链接概览
Table 1. Overview of expressive movement presets used in the study. 表 1.研究中使用的表现性动作预设概述。
Preset name 预设名称
Description 说明
Source/inspiration 来源/灵感
Video link 视频链接
1. Greeting 1.问候语
向观察者倾斜；停顿；上腔形成缓慢呼吸
Tilts toward the observer; pause; top chamber per-
forms slow breathing
Video 1 视频 1
2. Avoid 2.避免
向观察者倾斜；倾倒动作远离观察者；用上腔进行快速呼吸
Tilts toward the observer; tipping movement away
from the observer; fast breathing performed with
top chamber
Video 2 视频 2
3. Breathless 3.喘不过气
各腔缓慢呼吸；无运动；小幅充气；无运动；各腔快速呼吸
Slow breathing with all chambers; no movement;
small inflation; no movement; rapid breathing with
all chambers
16
Video 3 视频 3
4. Joyful 4.欢乐
Circular motion of the top of the robot while top 机器人顶部的圆周运动，同时顶部
Joyful dance 欢乐的舞蹈
5. Alarming 5.报警
两个底部气室充气，使机器人向前方倾斜；顶部气室快速呼吸（面向不同方向重复此序列 3 次
Two bottom chambers inflate and tip the robot for-
ward; top chamber breathes rapidly (repeats this
sequence 3 times facing different directions
Bird warning gestures 鸟类警告手势
Video 5 视频 5
6. Jellyfish 6.水母
缓慢呼吸，上腔和下腔之间稍有延迟
Slow breathing performed with slight delay between
top chamber and bottom chambers
Motions of a jellyfish 水母的动作
Video 6 视频 6
7. Frighten 7.惊吓
底腔充满空气；上腔由慢速呼吸转为快速呼吸
Bottom chambers filled with air; slow breathing
going to fast breathing on top chamber
Pufferfish defense 河豚防御
Video 7 视频 7
8. Sigh 8.叹息
所有腔室充气；无运动；完全放气
Inflation of all chambers; no movement; full defla-
tion
Sighing 叹息
Video 8 视频 8
2.3 Experiment procedure and stimuli 2.3 实验程序和刺激物
Due to Covid-19 restrictions, we conducted an online survey. Video recordings were made of each preset with the robot in front of a black background and used as stimuli (see Fig 1-right). First, participants watched a demonstration video of the robot Participants were allowed to watch each video of a preset multiple times before answering. For all videos, participants were asked "After watching the video, how did you perceive the robot?". For the first 3 movement presets, we asked participants to choose words from a predefined list, including the words matching their intended meanings. Participants could choose multiple words for each preset. For the remaining 5 exploratory presets, participants were asked to respond with single words freely chosen. The data collection approach was inspired by prior work using word associations. Word associations have been used as a means to evaluate conceptual structures and changes in attitude or belief within psychology and social science research 17. We adapt the method by using short expressive movement sequences, intended to convey a concept, state, or feeling, as stimuli, rather than a single word to gain access to spontaneous mental representations of what each preset signified. The questionnaire ended with four open-ended questions: "After having watched all the different videos of the robot, is there anything you would like to add?", "Did you compare the robot to something?", "Did the robot make you think of something?", and "How did the robot make you feel?". 由于 Covid-19 的限制，我们进行了在线调查。我们在黑色背景前录制了机器人的每个预设视频，并将其作为刺激物（见图 1-右）。首先，参与者观看机器人的演示视频 ，允许参与者在回答问题前多次观看每个预设视频。在所有视频中，参与者都会被问到 "观看视频后，您对机器人的感知如何？在前 3 个动作预设中，我们要求参与者从预定义列表中选择词语，包括符合其预期含义的词语。参与者可以为每个预设选择多个单词。在其余 5 个探索性预设中，我们要求参与者自由选择单个单词进行回答。这种数据收集方法的灵感来自于之前使用单词联想的研究。在心理学和社会科学研究中，词语联想一直被用作评估概念结构和态度或信念变化的一种手段17。我们对这一方法进行了调整，使用旨在传达概念、状态或感觉的简短表现性动作序列作为刺激，而不是单个单词，以获取每个预设符号所代表的自发心理表征。问卷最后有四个开放式问题："在观看了机器人的所有不同视频后，您还有什么要补充的吗？"、"您是否将机器人与某些东西进行了比较？"、"机器人是否让您想到了某些东西？"以及 "机器人给您带来了怎样的感受？"。
3 Results and Discussion 3 结果与讨论
3.1 Existing socially communicative movement presets 3.1 现有的社会交际动作预设
The distributions of words chosen by participants is shown in Fig. 2,Bottom. For all three presets, the word "Breathing" was the word most frequently chosen. We attribute this to the inflation used to actuate the robot. 参与者所选单词的分布情况如图 2 底部所示。在所有三种预设中，"呼吸 "是最常被选中的词。我们将其归因于用于驱动机器人的充气装置。
The word choices for each of the 3 existing socially communicative movement presets were to some extent overlapping with the intended meanings, however not unambiguous (see Fig. 2). To identify characteristic words for each preset, one may compare how many times a word was chosen for the specific preset in relation to the other presets, and for each preset to list the words that had a higher prevalence than for both of the other presets. 在现有的 3 个社会交际动作预设中，每个预设的选词在一定程度上都与预期含义重叠，但并不明确（见图 2）。为了确定每个预设的特征词，我们可以比较特定预设与其他预设的选词次数，并列出每个预设的选词率高于其他两个预设的选词率的词。
The only word that was chosen more frequently for the Breathless preset was "Breathless". Yet, this was not the word that was chosen the most for this preset, indicating that the movement was not predominantly understood as expressing breathlessness. 在 "喘不过气 "预设动作中，唯一一个被更频繁地选择的词是 "喘不过气"。然而，这并不是该预设动作中被选择最多的词，这表明该动作并不主要被理解为表达呼吸困难。
Compared to the two other presets, the Avoid preset scored higher for the words "Alarming", "Impulsive", "Dominating", "Frightening", " Aggressive", "Irritated", "Aroused", "Angry", "Excited", and "Afraid". These words all refer 与其他两个预设相比，"避免 "预设在 "惊恐"、"冲动"、"支配"、"恐惧"、"侵略性"、"烦躁"、"恼怒"、"愤怒"、"兴奋 "和 "害怕 "等词上得分更高。这些词都是指 The demonstration video of the robot is available at: 该机器人的演示视频可在以下网站观看： https://youtu.be/xI2RGWL6cXI. https://youtu.be/xI2RGWL6cXI。
to being in an intense, alert, and outgoing state, which is consistent with emotions felt when trying to avoid a person, or a situation. However, the valence of this state seems to have been ambiguous to participants, as both negative words (e.g., "Frightening" and "Afraid") as well as positive words ("Excited") were scored higher. 这与试图避开某人或某种情况时的情绪是一致的。然而，参与者对这种状态的价值取向似乎并不明确，因为消极词语（如 "害怕 "和 "恐惧"）和积极词语（"兴奋"）的得分都较高。
The Greeting preset scored higher for "Breathing", "Organic", "Alive", "Calm", "Submissive", "Depressed", and "Happy". Here again the valence of the emotional state seems to have been ambiguous as both "Depressed" and "Happy" were scored higher. The remaining words that were scored higher describe a nonthreatening, living being in a tranquil state, which is compatible with the dominant interpretation of the Greeting motion as signalling openness to social interaction found in prior work 15 问候 "预设在 "呼吸"、"有机"、"活着"、"平静"、"顺从"、"沮丧 "和 "快乐 "中得分较高。在这里，情绪状态的价态似乎又是模糊的，因为 "沮丧 "和 "快乐 "的得分都较高。其余得分较高的词语描述的是一种非威胁性的、处于宁静状态的生命体，这与之前研究中对问候语动作的主流解释一致15。
3.2 Exploratory movement presets 3.2 探索性运动预设
Word association distributions for the 5 exploratory presets are visualized as a stacked bar graph in Fig. 2 -Top. The graph includes all words that were mentioned 4 times, or more for one of the presets. Using the same strategy to identify characteristic words for each preset as above, we note that the Frighten preset was largely perceived as intended namely "Stressed", "Dominant" and "Aroused", Alarming was associated with activities that require higher breathing rates and intense emotions like "Working", "Heartbeat","Dancing", "Communicating" and "Searching", Joyful mostly triggered positive reactions like "Playful", "Joyful", "Dancing", and "Curious", while Sigh was confused with yawn (another paralinguistic respiration) and Jellyfish was associated to a relaxed, and calm breathing being. 5 个探索性预设的词语关联分布以堆叠条形图的形式直观地显示在图 2 - 上部。图中包含了所有被提及 4 次或更多次的预设词语。使用与上述相同的策略来识别每个预设的特征词，我们注意到，"惊吓 "预设在很大程度上被认为是 "紧张"、"支配 "和 "唤醒"，而 "惊吓 "则与 "工作 "等需要较高呼吸频率和强烈情绪的活动相关、而 "叹息 "则与打哈欠（另一种副语言呼吸）相混淆，"水母 "则与放松和平静的呼吸相关联。
3.3 General commments 3.3 一般性评论
We received 44 general comments. 11 were translated from Danish to English by a native speaker. To prepare the text corpus for analysis with the Voyant Tools open text mining environment 18 we removed common word endings like "-ing", "-ed","-s" to make our results more accurate. We chose to analyze the comments via text mining in order to have a quantitative and more objective approach to a completely subjective input. Figure 5 shows the collocates -directed networkgraph of higher frequency keywords and terms that appear in proximity. 我们收到了 44 条一般性意见。其中 11 条是由母语为丹麦语的人从丹麦语翻译成英语的。为了准备使用 Voyant Tools 开放式文本挖掘环境 18 分析文本语料库，我们删除了"-ing"、"-ed"、"-s "等常见词尾，以使结果更加准确。我们选择通过文本挖掘对评论进行分析，以便对完全主观的输入采用定量和更客观的方法。图 5 显示了出现频率较高的关键词和相近术语的同义词导向网络图。
The collocates graph of the combined general comments indicate that the robot was perceived either as a jellyfish or animal-like sea creature, or as a human body part (specifically lungs, belly, or heart) and as communicating certain distinct emotional states. 综合一般性评论的搭配图显示，机器人要么被视为水母或类似动物的海洋生物，要么被视为人类的身体部位（特别是肺、腹部或心脏），并被视为在传达某种独特的情感状态。
4 Future work 4 今后的工作
As future work, we plan to develop more refined motion presets for the robot based on insights gained from this pilot study. In addition, we will integrate physical 在未来的工作中，我们计划根据此次试点研究中获得的见解，为机器人开发更精细的运动预设。此外，我们还将整合物理
Fig. 2. Clustered (bottom) and stacked (top) bar charts showing the distribution of the words chosen for the presets. Bottom: Existing socially communicative movement presets. Top: Words with higher frequency for exploratory movement presets. 图 2.显示预设所选词语分布的聚类（下）和堆叠（上）条形图。下图现有的社会交际动作预设。上图：探索性动作预设中出现频率较高的词语。
Fig. 3. Collocates graph of the general comments representing the main keywords in blue and their main collocates (up to 7 words in proximity) in orange. 图 3.一般性评论的同义词图，蓝色代表主要关键词，橙色代表其主要同义词（最多相邻 7 个词）。
sensors and computer vision to make the robot interactive, and explore adding additional NVC modalities including sound and light, to obtain both clearer and more complex communication, and plan to test whether they may contribute to e.g. communicating valence more clearly. Additional physical experiments are also required to elucidate the soft robot's potential for haptic interaction. 此外，还需要进行更多的物理实验，以阐明软体机器人在触觉互动方面的潜力。还需要进行更多的物理实验，以阐明软体机器人在触觉互动方面的潜力。
References 参考资料
Mehrabian, A. Nonverbal communication (Routledge, 2017). Mehrabian，A.《非语言交流》（Routledge，2017）。
Cha, E., Kim, Y., Fong, T., Mataric, M. J., et al. A survey of nonverbal signaling methods for non-humanoid robots. Foundations and Trends in Robotics 6, 211-323 (2018). Cha, E., Kim, Y., Fong, T., Mataric, M. J., et al.机器人学基础与趋势》 6, 211-323 (2018).
Poyatos, F. The Morphological and Functional Approach to Kinesics in the Context of Interaction and Culture. 20, 197-228 (1977). Poyatos, F. The Morphological and Functional Approach to Kinesics in the Context of Interaction and Culture.20, 197-228 (1977).
Saunderson, S. & Nejat, G. How Robots Influence Humans: A Survey of Nonverbal Communication in Social Human-Robot Interaction. International Journal of Social Robotics 11. Company: Springer Distributor: Saunderson, S. & Nejat, G. How Robots Influence Humans：人机交互中的非语言交流调查》。国际社会机器人学杂志》第 11 期。公司：SpringerSpringer Distributor：
