Introduction 介绍
Social touch interactions add nuance to our communication – a light squeeze on an anxious patient's arm calms them; a firm handshake asserts trust in a newly struck business deal; even a light tap on the shoulder can increase trust and cooperation between strangers [1]. We communicate comfort, love, and safety through touch, promoting pro-social behaviours and forging deep emotional bonds that help form and maintain relationships [2], [3].
社交触摸互动为我们的沟通增添细微差别 - 对焦虑的患者轻轻握住胳膊可以使他们平静;坚定的握手表明对新达成的商业交易的信任;甚至在陌生人之间轻轻拍拍肩膀也能增加信任和合作 [1] 。我们通过触摸传达舒适、爱和安全感,促进亲社会行为,建立深厚的情感纽带,有助于形成和维护关系 [2] , [3] 。
It is increasingly common for partners, family members, and close friends to be separated e.g., due to professional, academic, military, health responsibilities [4], [5], fueling a growing appetite for machine-mediated social touch [6], [7] that can re-introduce valuable touch-based interactions where natural person-to-person contact is not practical [8].
伴侣、家人和亲密朋友因职业、学术、军事、健康责任等原因而分离变得越来越普遍,这加剧了对机器介导社交触感的需求,可以重新引入有价值的基于触感的互动,以弥补自然人际接触不可行的情况。
How we perceive a communicated sentiment [9], [10] can be heavily influenced by the pre-existing relationship. Natural interactions take place within complex ecosystems of history, condition, and purpose, all of which color the encoding of emotional perception [11]. Studying how these interactions are received and interpreted must include the context and relationship they exist in [12]. This is certainly true of touch: e.g., touch between strangers is unlikely to be interpreted as surprise, envy, or pride [13], [14].
我们如何感知传达的情绪 [9] , [10] 可能会受到先前关系的重大影响。自然互动发生在复杂的历史、条件和目的生态系统中,所有这些都影响情绪感知的编码 [11] 。研究这些互动如何被接受和解释必须包括它们存在的背景和关系 [12] 。这在触摸方面当然也是如此:例如,陌生人之间的触摸不太可能被解释为惊讶、嫉妒或骄傲 [13] , [14] 。
People are capable of affectively interpreting simple notification-style tactile sensations [7], [15], [16]; in fact, it is natural to comprehend signals like high frequency choppy buzzing as urgent irritation, or soft rolling rumbles as calming reassurance [17]. Further, haptic animation displays have been embedded into a chair for on-back interaction to incorporate multimodal immersion for visual media [18] and incite a number of intriguing experiences where discrete tapping sensations simulate rain, or low rumblings evoke the purring of a big cat [19]. For this work, we test the feasibility of vibrotactile animation for haptic messaging. To leverage these sensations on a wearable, we ask: can partners with a shared context and history convey high-resolution emotional information, using just low-resolution spatial vibration through a relatively simple vibrotactile animation display?
人们能够有效地解释简单的通知式触觉感觉;事实上,理解高频刺耳的嗡嗡声为紧急刺激,或柔和的隆隆声为安抚的安慰是很自然的。此外,触觉动画显示已嵌入到椅子上,用于背部互动,以融合视觉媒体的多模感官沉浸,并引发一系列有趣的体验,其中离散的轻拍感觉模拟雨,或低沉的隆隆声唤起大猫的呼噜声。对于这项工作,我们测试了用于触觉信息传递的振动触觉动画的可行性。为了利用这些感觉在可穿戴设备上,我们问:在共享背景和历史的伙伴能够仅通过相对简单的振动触觉动画显示传达高分辨率的情感信息吗?
To test the feasibility of vibrotactile animation for haptic messaging, we prototyped a wearable haptic display, scaling a large chair-sized interface [19] down to an array of 8 small tactors to fit along the forearm (see Fig. 1). The accompanying message design interface uses the exact principles developed by [20]: users define the sensation by directly drawing on a touchscreen and a continuous tactile signal (i.e., without unintentional segmentation or path break) is interpolated spatiotemporally along the drawn curve. To assess the potential for affective interpersonal but remote haptic communication, we devised a three-part study (summarized in Fig. 2), as well as a device validation pilot, in which we:
为了测试触觉信息传递的振动触觉动画的可行性,我们制作了一个可穿戴的触觉显示原型,将一个大型椅子大小的界面缩小到一个由 8 个小触觉器组成的阵列,以适应前臂(见图 1)。伴随的消息设计界面使用了由 [20] 开发的确切原则:用户通过直接在触摸屏上绘制来定义感觉,并且连续的触觉信号(即,没有意外分段或路径中断)在绘制的曲线上进行时空插值。为了评估情感间但远程触觉交流的潜力,我们设计了一个三部分研究(见图 3 概述),以及一个设备验证试点,在该试点中我们:
Built an 8-tactor wearable prototype and conducted a small pilot on
people to evaluate the feasibility of haptic messaging for affect communication.Npilot=12
建立了一个 8-马达可穿戴原型,并对 人进行了小规模试点,以评估触觉消息传递在情感沟通中的可行性。Npilot=12 Surveyed
people about messages they send to people they want to maintain touch relationships with despite obstacles such as distance or health issues. From this data, we constructed 10 scenarios that capture realistic context which might naturally prompt touch as a communicative element.Nsurvey=201
对 人进行了调查,了解他们向希望保持联系关系的人发送信息的情况,尽管存在距离或健康问题等障碍。根据这些数据,我们构建了 10 个场景,捕捉了可能自然促使触摸成为沟通要素的现实背景。Nsurvey=201 Collected haptic message designs by 10 dyads (
individuals) in close co-habitation relationships for their partners. These messages are contextualized by scenario prompts and include a personal wild card message of their choice.Ndesign=20
10 对( 个个体)在密切共同生活关系中为他们的伴侣设计的触觉信息。这些信息由情境提示进行背景化,并包括他们选择的个人王牌信息。Ndesign=20 Collated interpretations from
individuals who experienced haptic messages designed by strangers, their partners, and themselves a week earlier.Ninterpret=21
名个体的整理解释,他们在一周前体验了陌生人、伴侣和自己设计的触觉信息。Ninterpret=21
We assessed the physical aspects of the haptic message designs from (3) by intended emotion, identified features offering the greatest insight, then incorporated these into a machine learning model predicting emotion from message. We report how machine recognition of the emotional scenario prompts compared to that of human interpretations from (4), broken down by the interpreter's relationship to designer: stranger, partner, or self (ordered in overall increasing interpretation rate). We describe the strategies that participants took in designing and interpreting encodings, noting where partner-focused strategies perform better than non-partnered counterparts; and suggest improvement priorities for the next iteration of a haptic messaging prototype. Finally, we observe how the interaction experience excited a spirit of play and whimsicality in design and recognition – an intuitive key in unlocking the privately shared tactile language between partners.
我们通过预期情感评估了来自(3)的触觉信息设计的物理方面,确定提供最深入见解的特征,然后将这些特征纳入一个机器学习模型,用于从信息中预测情感。我们报告了机器对情感场景的识别与(4)中人类解释的比较,按照解释者与设计者的关系进行了拆分:陌生人、伴侣或自己(按照整体解释率递增的顺序排列)。我们描述了参与者在设计和解释编码时采取的策略,指出伴侣关注的策略在某些方面优于非伴侣的对应策略;并建议改进下一代触觉信息传递原型的优先事项。最后,我们观察到互动体验如何在设计和识别中激发了一种游戏和奇思妙想的精神 - 这是解锁伴侣之间私密共享的触觉语言的直观关键。
Overall, this paper contributes:
总的来说,本文的贡献是:
a compilation of results from human and machine recognition of emotion-based intent in haptic messages;
人类和机器在触觉信息中识别基于情感意图的结果编译;evidence that shared history influences the interpretation of playful affective haptic messages;
共享历史影响对玩味情感触觉信息的解释的证据;a summary of design strategies and engineering parameters of haptic messages created by and for partners in close relationships, and a synthesis of our results into insights to inform future systems to support effective interpersonal haptic messaging.
合作伙伴间创造和使用的触觉信息的设计策略和工程参数总结,以及我们的研究结果综合成见解,以指导未来支持有效人际触觉信息传递系统。
Background 背景
When studying machine-mediated haptic expressions of emotion, we want the touch to be representative of a genuine affective experience [9], [21]. In this work, we are further interested specifically in purposeful emotive interactions [22]. Thus, we present relevant related work and explain how it has informed our approach in two parts: (1) machine-mediated touch interaction where participants are (2) grounded in real emotions rooted in familiar events generating authentic touch expressions.
在研究机器介导的触觉情感表达时,我们希望触摸能够代表真实的情感体验。在这项工作中,我们对有目的的情感互动特别感兴趣。因此,我们提出相关的相关工作,并解释它如何在两个方面指导我们的方法:(1)机器介导的触摸互动,参与者根植于真实情感,这些情感源于熟悉事件,产生真实的触摸表达。
Machine-Mediated Touch and Display Expressivity.
Defined as “the ability of one actor to touch another actor over a distance by means of tactile or kinesthetic feedback technology,” machine-mediation differs from direct touch where actors physically experience and reciprocate social touch in-person [23] (p153). There are many wearable or handheld devices (both research prototypes and commercial products) that purport to bridge physical distance to enable social touch [6]. The form of touch varies: the Hey bracelet1 uses squeeze sensations (a motor rolls to tighten the band); Shaker [24] transmits a shake via a current between connected solenoids; the research tool, The Tactile Emoticon System is a glove form factor that transmits and receives pressure, heat, and vibration, concluding that interpretative value may hinge on message personalization [25]. Since studies have shown that users can infer nuanced affective information from simple vibrations alone [23], [26], [27], we inspect whether affective meaning can be made through custom designed vibrotactile messages.
机器介导的触觉和显示表现性。定义为“一个演员通过触觉或动觉反馈技术远程触摸另一个演员的能力”,机器介导与直接触摸不同,直接触摸是指演员在现场亲身体验和回应社交触摸(第 153 页)。有许多可穿戴或手持设备(包括研究原型和商业产品)声称可以弥合物理距离以实现社交触摸。触摸形式各异:Hey 手环使用挤压感觉(电机滚动以收紧手环);Shaker 通过连接的电磁铁之间的电流传输震动;研究工具——触觉表情符号系统是手套形式,传输和接收压力、热量和振动,得出结论认为解释价值可能取决于消息个性化。由于研究表明用户可以仅通过简单的振动推断微妙的情感信息,我们检查是否可以通过定制设计的振动触觉消息传达情感含义。
Haptic Spatial Animation to Leverage Expressive Sketching.
Haptic animation [18] has demonstrated that perceptually interpolated (i.e., animated) spatial vibrotactile display creates an intriguingly varied and rich design space [20], [28]. Tactor arrays have been used to create interesting effects. From early attempts to discern simple directional lines [29], to simulations of real-world haptic experiences like a snake crawling up an arm [30] or a cat walking across one's back [18], the field has made great strides in approximating convincing haptic effects from simple vibrations.
触觉空间动画以提升表现性素描。触觉动画已经证明,感知插值(即动画)的空间振动触觉显示创造了一个引人入胜且丰富多样的设计空间。触觉器阵列已被用于创造有趣的效果。从早期试图辨别简单方向线条,到模拟现实世界的触觉体验,如蛇爬上手臂或猫走过背部,该领域在从简单振动中逼真模拟令人信服的触觉效果方面取得了长足进展。
The density and positioning of the tactor array depends on the sensitivity of the body part stimulated. For instance, [31] developed a 3x3 tactor array that was sufficient for the entire back to feel fully activated; while [20] used a trapezoidal 5-tactor array embedded in a chair. A forearm is much more sensitive with two-point discrimination (the minimum distance where two distinct points can be differentiated) recorded at about 30.7-45.4 mm from 43 subjects [32]), so devices need not be more dense than this linear distance. Distinct excitations were distinguishable only about 30-40% of time (chance 14.3%) at 25 mm apart [33], suggesting that most vibrations from neighbouring tactors within this distance may be experienced as a continuous (i.e., without segmentation). At 22-44 mm linear tactor distances, an illusion could be created of a snake moving in various ways across the arm [30]. We built our display to be within the two-point discrimination range [32], similar in range as [30].
触觉器阵列的密度和位置取决于被刺激的身体部位的敏感度。例如, [31] 开发了一个 3x3 的触觉器阵列,足以让整个背部感受到充分的激活;而 [20] 使用了嵌入在椅子中的梯形 5 触觉器阵列。前臂对两点辨别(即可以区分两个不同点的最小距离)更为敏感,43 名受试者记录的距离约为 30.7-45.4 毫米 [32] ,因此设备无需比这个线性距离更密集。在 25 毫米间隔处,只有约 30-40%的时间可以区分不同的激活(机会为 14.3%) [33] ,这表明在这个距离内来自相邻触觉器的大多数振动可能会被感受为连续的(即没有分割)。在 22-44 毫米的线性触觉器距离下,可以制造出一种蛇在手臂上以各种方式移动的幻觉 [30] 。我们的显示器建造在两点辨别范围内 [32] ,与 [30] 的范围相似。
The tactor array needs design tools for building haptic animations into socio-affective touch. Given that even relatively low-fidelity sensations can be emotively expressive [15], context may be as important to interpretation as the sensation itself [34]. Therefore, we explored the impact of context, and privilege a sender's design experience over high fidelity display. Specifically, we anticipate that access to a spatiotemporal design palette will allow participants to define vibrotactile messages with greater personal significance with the potential for novel haptic experiences and expressions. We draw inspiration from the haptic design palette presented by [19] and affective vibrotactile parameters proposed by [26], [27], to develop an accompanying haptic sensation editor.
触觉阵列需要设计工具,用于将触觉动画构建到社会情感触觉中。考虑到即使是相对低保真度的感觉也可以表达情感 [15] ,环境可能与感觉本身一样重要。因此,我们探讨了环境的影响,并优先考虑发送者的设计经验而不是高保真度显示。具体来说,我们预计访问时空设计调色板将使参与者能够定义具有更大个人意义的振动触觉信息,从而产生新颖的触觉体验和表达。我们从 [19] 提出的触觉设计调色板和 [26] , [27] 提出的情感振动触觉参数中汲取灵感,以开发配套的触觉感觉编辑器。
Emotion-Related Remote Touch Between Strangers.
With direct human-to-human touch established as a medium for emotion content [35], it follows to ask how much of this emotion encoding and decoding is retained when direct touch is intercepted by another medium or device. [36] had participants generate emotion-laden handshakes using a commercially available force-feedback joystick and found that those sensations were human-interpretable at roughly twice that of chance (33%, where chance was 1 in 7 or 14%).
陌生人之间的情感相关远程触摸。随着直接的人际接触被确定为情感内容的媒介 [35] ,接下来要问的是,当直接触摸被另一个媒介或设备拦截时,有多少情感编码和解码被保留。 [36] 让参与者使用一款商用力反馈操纵杆生成带有情感的握手,发现这些感觉在人类解读方面的准确率大约是随机机会的两倍(33%,其中机会是 7 分之 1 或 14%)。
Even without direct device contact, haptic sensations can communicate emotional content. The UltraHaptics system sends ultrasonic air pressure waves to deliver tactile sensations mid-air. [37] asked participants to design sensations that represent the emotions elicited by a provided picture, by modulating frequency, duration, and intensity. Another set of participants then rated how well suited some 10 haptic sensations were to a given picture. Again, there is evidence that mediated social touch can communicate emotion between people: participants consistently rated the haptic description designed with the picture with “high appropriateness”. Here we ask: since close relationships create more opportunity for communicating through touch, what is the difference in emotion recognition between strangers versus that of close partners?
即使没有直接设备接触,触觉感觉也能传达情感内容。UltraHaptics 系统通过发送超声波气压波来传递空中触觉感觉。 [37] 要求参与者设计代表所提供图片引发的情绪的感觉,通过调节频率、持续时间和强度。另一组参与者随后评价了一些 10 种触觉感觉对于给定图片的适用程度。再次,有证据表明,通过媒介社交触觉可以在人与人之间传达情感:参与者一致地评价了与图片设计的触觉描述的“高适宜性”。在这里我们问:由于亲密关系为通过触觉传达情感创造了更多机会,陌生人与亲密伴侣之间的情感识别有何不同?
Relationship as Context.
While most people would intuitively accept that humans can communicate emotion through touch, it is still somewhat surprising when sentiments like anger, love, gratitude can be recognized at rates above chance by strangers in a lab directly touching one anothers’ forearm [13]. Certain emotions have better recognition rates in the US versus in Spain; cultural relationship may explain some of these differences [13]. Studies on facial emotion recognition found that mutual cultural membership adds contextual background for how an expression may be made [38] which in turn influences emotion interpretation. Our interpretation of touch is similarly influenced [39] wherein culture defines who, when, where, and how we touch one another. So what happens where touch history extends beyond being simply cultural? Turns out that even in machine-mediated touch, relationship context (e.g., are we partners, friends, work colleagues?) is crucial for generating and interpreting Tactile Emoticons [25].
关系作为背景。虽然大多数人直觉地认为人类可以通过触摸传达情感,但当陌生人在实验室直接触摸彼此的前臂时,能够以高于偶然率的速度识别愤怒、爱、感激等情感时,仍然有些令人惊讶。在美国,某些情绪的识别率比在西班牙高;文化关系可能解释其中一些差异。关于面部情绪识别的研究发现,共同的文化成员资格为表达方式可能被制定提供了背景 [38] ,从而影响情感的解释。我们对触摸的解释同样受到影响 [39] ,在这方面,文化定义了我们如何、何时、何地以及如何触摸彼此。那么,当触摸历史超越简单的文化时会发生什么?事实证明,即使在机器介导的触摸中,关系背景(例如,我们是伴侣、朋友、同事?)对于生成和解释触觉表情符号至关重要 [25] 。
Many more factors contribute to the contextual framework that ultimately informs how a touch between two people is perceived [11]: the relationship between them, the events triggering the touch contact, the environment and backdrop, and each participants’ comfort with emotional expression. With respect to relation, Thompson et al. examined touch interaction between couples and found that partners were better able to distinguish typically self-focused emotions like embarrassment, envy, pride than strangers [14]. Since technologically-mediated touch seems to follow common patterns for relationship contextualized haptic interaction [25], we wonder if recognition improves with relationship closeness where emotion-based touch messages authored by the participant themselves, their partner, and strangers may be successively less interpretable. Quantifying reasons for the difference in recognition rates influences how we structure our touch communication systems and interaction design.
许多因素共同构成了最终决定两人之间触摸如何被感知的背景框架:他们之间的关系、触摸接触引发的事件、环境和背景,以及每个参与者对情感表达的舒适程度。关于关系,汤普森等人研究了夫妻之间的触摸互动,发现伴侣们能够更好地区分通常以自我为中心的情绪,如尴尬、嫉妒、自豪,而不是陌生人。由于技术介导的触摸似乎遵循了关系情境化触觉互动的共同模式,我们想知道在关系亲密度增加时,是否会提高识别能力,其中由参与者自己、他们的伴侣和陌生人创作的基于情感的触摸信息可能会逐渐变得不那么可解释。量化识别率差异的原因影响我们如何构建触摸通信系统和交互设计。
Materials and Methods 材料和方法
A consumer-ready haptic messaging device would require careful iteration over hardware, functionality, and user experience. Here, we demonstrate a proof of concept for encoding emotional content into haptic animation – a necessary first step (process summary in Fig. 2).
消费者可用的触觉消息传递设备需要对硬件、功能和用户体验进行仔细的迭代。在这里,我们展示了将情感内容编码到触觉动画中的概念验证 - 这是一个必要的第一步(流程摘要见图 2 )。
All experiments were conducted in 2019 and in accordance with the organization's ethical policy regarding human participant testing (with protocol as conducted later approved by WIRB, ref# AGHM-2019). Environmental Health and Safety approved the device prior to the study.
所有实验均在 2019 年进行,并遵守组织关于人类参与者测试的道德政策(随后进行的协议已获得 WIRB 批准,参考编号为 AGHM-2019)。环境健康与安全在研究之前批准了该设备。
A. The Haptic Display A. 触觉显示
We describe the specifications of our haptic animation prototype and the interface for designing the sensations.
我们描述了我们的触觉动画原型的规格以及设计感觉的界面。
1) Building the Prototype
1) 建立原型
Our prototype's custom haptic display features eight voice-coil vibrotactile tactors (model: TEAX13C02, Tectonic Elements, U.K.2) which are arranged in equilateral triangles (35 mm sides) along two columns, and padded with laser-cut Polyurethane foam (Fig. 1(b)). Each tactor is housed in a 3D-printed casing to isolate electrical components from directly touching the skin, and covered with an insulated cover. The vibrating element of each actuator is covered with a thin 15 mm diameter disk that contacts the skin. The actuators are computer-controlled using an audio interface (Motu, USA, model 24Ao2) and powered with a set of audio amplifiers (MAX983062).
我们的原型定制触觉显示器采用了八个音圈振动触觉执行器(型号:TEAX13C02,Tectonic Elements,英国),这些执行器排列成等边三角形(边长 35 毫米)沿着两列,并用激光切割的聚氨酯泡沫垫(图 1)。每个执行器都被放置在一个 3D 打印的外壳中,以隔离电子元件直接接触皮肤,并覆盖有绝缘罩。每个执行器的振动元件上覆盖有一个直径为 15 毫米的薄圆盘,与皮肤接触。这些执行器通过音频接口(Motu,美国,型号 24Ao)进行计算机控制,并由一组音频放大器(MAX98306)供电。
2) Tactor Position 2) 触发器位置
The inner forearm from wrist to elbow is tactually sensitive, socially discrete, convenient and practical, without hindering the hand [40], [41]. These traits make it an excellent candidate for placing a tactile display. To leverage the wide design space of haptic animation [18], [19], we followed [20]'s blueprint to create a medium-fidelity prototype of a haptic animation display with tactors positioned as vertices of equilateral triangles but with intertactor distance scaled down to 35 mm to be wearable on the arm (rather than embedded in a chair-back as in [20]). This distance was so chosen to fall within the two-point discrimination range of 30.7 mm - 45.4 mm for the forearm [32] in order to render a continuous vibration sensation between adjacent tactors. When engaged in the frequency range of 20-300 Hz, the tactor array on the device has a conservative active surface area that covers the contact area of the device at about 60 mm wide by 155 mm long (since each tactor has a two-point discrimination radius of 30 mm). Strapped tightly to an adult forearm, the device casing that houses the tactors has a height of 20 mm over the contact area.
从手腕到肘部的前臂在触觉上敏感,社交上离散,方便实用,不妨碍手部。这些特点使其成为放置触觉显示器的绝佳选择。为了利用触觉动画的广阔设计空间,我们遵循了某人的蓝图,创建了一个中等保真度的触觉动画显示器原型,其中触觉器被放置在等边三角形的顶点上,但触觉器之间的距离缩小到 35 毫米,以便佩戴在手臂上(而不是像某人那样嵌入在椅背上)。选择这个距离是为了落在前臂的两点辨别范围 30.7 毫米至 45.4 毫米之内,以便在相邻触觉器之间产生连续的振动感觉。当在 20-300 赫兹的频率范围内运行时,设备上的触觉器阵列具有保守的有效表面积,大约覆盖设备的接触区域,宽 60 毫米,长 155 毫米(因为每个触觉器的两点辨别半径为 30 毫米)。紧贴在成年人前臂上的设备外壳,内置触觉器,接触区域高度为 20 毫米。
3) Message Editor & Process
3) 消息编辑器和流程
So that lay designers could access this prototype function with minimal learning, we developed a rudimentary graphical user interface (GUI) in which a designer can define their haptic sensations by manipulating a set of vibration parameters (summarized in Table I) and drawing directly on the representative display area of a touch screen (Fig. 1(a)).
为了让非专业设计师能够在最小学习成本下访问这个原型功能,我们开发了一个基本的图形用户界面(GUI),设计师可以通过操纵一组振动参数(总结在表 I 中)并直接在触摸屏的代表性显示区域上绘制来定义他们的触觉感觉(见图 1(a) )。
In playback, the resulting sensation is graphically presented by a circle that follows a designer-laid track, while playing out tactually on the arm in the same timeline.
在回放中,产生的感觉通过一个圆形图形呈现,该圆形沿着设计师设置的轨道移动,同时在同一时间轴上在手臂上触觉上展现。
Terminology.
We refer to the touch screen region as the drawing surface where designers can define the size of the circle or brush to draw a track defining the path that the haptic sensation travels. A track can consist of one or more strokes which are continuous drawn segments. The circular brush's diameter represents how wide the track feels – e.g., a large brush radius signifies a wider or thicker track line such that tactors passing under the brush are activated. Upon playback, participants feel the haptic animation which is the sensation of the recorded design. A haptic encoding refers to a tactile animation designed to communicate a specified intent; this haptic signal together with the intent are a haptic message.
术语。我们将触摸屏区域称为绘图表面,设计师可以在其中定义圆圈或画笔的大小,以绘制定义触觉感觉传播路径的轨迹。轨迹可以由一个或多个连续绘制的线段组成。圆形画笔的直径代表轨迹的宽度感觉 - 例如,较大的画笔半径表示更宽或更厚的轨迹线,使得通过画笔下方的触觉器被激活。在回放时,参与者感受到触觉动画,即记录设计的感觉。触觉编码指的是旨在传达特定意图的触觉动画;这种触觉信号与意图一起构成了触觉信息。
Parameters Editable Through the GUI (Table I).
As well as spatial path and dynamics, the designer can modify vibration intensity by editing the signal waveform's amplitude and frequency (both defined before drawing a message that does not vary over the course of a single message). Brush size refers to the circle diameter (i.e., width of the drawn track), such that tactors within the brush's circular boundary are activated at varying intensities based on diffusion type. Three diffusion types (linear, quadratic, exponential) allow users to define how sharp and focused the animation brush feels across the path's breadth during tactile replay. With linear diffusion, there is a gradual fade from the ball's centre to its border; exponential causes the most dramatic fade with most of the sensation near the path centerline; quadratic is in between.
通过 GUI 可编辑的参数(表 I )。除了空间路径和动态外,设计师还可以通过编辑信号波形的振幅和频率来修改振动强度(在绘制消息之前定义,不会在单个消息的过程中变化)。刷子大小指的是圆直径(即绘制轨迹的宽度),因此刷子圆形边界内的触觉器根据扩散类型以不同强度激活。三种扩散类型(线性、二次、指数)允许用户定义在触觉重播期间动画刷子在路径宽度上的感觉有多锐利和集中。线性扩散使球的中心到边界逐渐消失;指数使感觉大部分集中在路径中心线附近,呈现最戏剧性的消失;二次则介于两者之间。
Drawing Process. The drawn track defined the x-y coordinate of the 2D drawing surface as well as the time variation of the stroke. After drawing a track, users can then record their haptic message, play it back, and edit the sensation using all available parameters including speeding up or slowing down the animation along the existing track. Any edits to track placement requires a new design.
绘图过程。绘制的轨迹定义了 2D 绘图表面的 x-y 坐标,以及笔画的时间变化。绘制轨迹后,用户可以录制他们的触觉消息,播放它,并使用所有可用参数编辑感觉,包括加快或减慢沿着现有轨迹的动画。对轨迹位置的任何编辑都需要新的设计。
Our priority was for designers to achieve their haptic-message vision. Although we tried to make the GUI usable, its success was not a focus at this stage, so a researcher helped designers navigate the interface.
我们的首要任务是让设计师实现他们的触觉信息愿景。尽管我们努力使 GUI 可用,但在这个阶段,成功并不是重点,因此一位研究人员帮助设计师浏览界面。
4) Understanding the Display: A Pilot Study
4) 理解显示:一项试点研究
To ensure that participants could tactually perceive the overall physical sensations rendered by our haptic display (a prerequisite of interpreting their intended meaning), we conducted a pilot in which we asked participants to re-draw eight researcher-defined haptic animations, so chosen to vary shape, area covered, segment count, direction, curvature and angles.
为了确保参与者能够触觉感知我们的触觉显示呈现的整体物理感觉(解释其预期含义的先决条件),我们进行了一项试点研究,要求参与者重新绘制八个研究者定义的触觉动画,这些动画被选择以变化形状、覆盖面积、段数、方向、曲率和角度。
To reduce novelty effects of the device, participants who received the stimuli from the haptic display were first introduced to the prototype in a sandbox session where they could experiment with the controls and sensations. We asked them to find and tell us the parameter ranges on brush size, diffusion, amplitude and frequency where haptic sensation felt both clear and comfortable; then set participant-specific ranges based on their preferences. In general, pilot participants found the sensation most pleasant at low vibrational frequencies, reporting a range of
为了减少设备的新奇效应,从触觉显示器接收刺激的参与者首先在沙盒会话中介绍了原型,他们可以在那里尝试控件和感觉。我们要求他们找到并告诉我们关于刷子大小、扩散、振幅和频率的参数范围,使触觉感觉清晰且舒适;然后根据他们的偏好设置参与者特定的范围。总体而言,试飞参与者发现在低振动频率下感觉最愉悦,报告了 31.7 Hz(11.7)至 91.2 Hz(25.6)的范围。
To assess device rendering accuracy as compared to actual continuous touch contact,
为了评估设备呈现的准确性与实际连续触摸接触相比,
5) Outcomes 5) 结果
As summarized in Table II, participants correctly distinguished the segment count (device
如表 II 所总结,参与者正确区分了段数(设备
B. Message Meaning Phase: MTurk Online Survey
B. 信息含义阶段:MTurk 在线调查
To build a realistic set of message meanings (i.e., situated within scenarios) for which participants would design descriptive haptic encodings, we surveyed individuals who had been in at least one long-distance relationship (
为了构建一个现实的消息含义集合(即,嵌入在情境中),供参与者设计描述性触觉编码,我们对至少参与过一次远距离恋爱关系的个体进行了调查(
Survey respondents (mean age 33.4 years) reported on relationships with a spouse or romantic partner (68.2%), parent (12.9%), friend (10.9%), child (3.5%), grandparent (2.5%) and sibling (2.0%), most of which (54.5%) were long-distance for at least a year. We compiled their free-form responses to “What message would you most like to send to your loved one? (all of which were conversation initiations). Two independent raters looked for the main themes of the intended ensuing conversation; resolving the two independent lists, we agreed on the 8 categories listed in Table III. The most common category involved bids for conversation without a specific topic, including general updates that often open with Hey. Next are a series of emotions elicited: excitement, miss you (sometimes also referred to as longing), sadness, love, anger, gratitude, anxiety. We found that some messages expressed high urgency or arousal, through use of all-caps (e.g., “IS SOMETHING WRONG?!”) so we noted the extremes as calm, attention. For each category (plus calm and attention), we created scenario prompts to ensure common contexts with implicit roots in emotion. Finally, since each partner pair has a distinct communication style and background, we added a wildcard message scenario for a total of 11 prompts for message generation listed in Table IV.
调查对象(平均年龄 33.4 岁)报告了与配偶或恋人(68.2%)、父母(12.9%)、朋友(10.9%)、子女(3.5%)、祖父母(2.5%)和兄弟姐妹(2.0%)的关系,其中大多数(54.5%)至少有一年的远距离关系。我们整理了他们对“您最想传达给您所爱的人的信息是什么?”的自由形式回答(所有这些都是对话的开始)。两位独立的评分员寻找了预期对话主题的主要主题;整理两个独立的列表后,我们就表 III 中列出的 8 个类别达成了一致。最常见的类别涉及对话请求而没有特定主题,包括通常以“嘿”开头的一般更新。接下来是一系列引发情感的情绪:兴奋、想念你(有时也称为渴望)、悲伤、爱、愤怒、感激、焦虑。我们发现一些信息表达了高度的紧急性或唤醒,通过使用全大写字母(例如,“有什么问题吗?!”),因此我们将极端情况记录为冷静、关注。对于每个类别(以及冷静和关注),我们创建了情景提示,以确保具有情感根源的常见背景。 最后,由于每对合作伙伴都有独特的沟通风格和背景,我们为消息生成添加了一个通配符消息场景,总共列出了 11 个提示,详见表 IV 。
C. Design Phase: In-Person Dyads
C. 设计阶段:面对面二人组
We recruited ten dyads (self-reported as 9 male, 11 female; aged
我们招募了十对(自称 9 男 11 女;年龄为
1) Familiarization 1) 熟悉
After getting comfortable with the device and controls, participants proceeded to perceiving and distinguishing a useful range of sensations. To reduce novelty effects and establish a message sending / receiving experience, dyads were first given a chance to play with the device together. Specifically, we alloted time for: (a) Sandbox mode to establish a messaging context and reduce novelty effects, consisting of 10-15 min of playing with the device and sending instant messages, wherein Person A draws on the touch screen while Person B wears the display and vice versa; (b) a sensitivity test so researchers can ensure participants are able to perceive basic encoding elements; (c) a learning phase for designers to familiarize themselves with the GUI and the device capability.
在熟悉设备和控制器后,参与者开始感知和区分一系列有用的感觉。为了减少新奇效应并建立消息发送/接收体验,首先让双人组合有机会一起玩这个设备。具体来说,我们安排了以下时间:(a) 沙盒模式用于建立消息传递背景和减少新奇效应,包括与设备互动并发送即时消息,人员 A 在触摸屏上绘图,人员 B 戴着显示器,反之亦然,持续 10-15 分钟;(b) 灵敏度测试,以确保研究人员能够感知基本的编码元素;(c) 设计师学习阶段,熟悉 GUI 和设备功能。
One participant wore the haptic display device on their forearm while the other drew on the touch screen interface to transmit real-time messages. They switched roles halfway through the sandbox session; we gently suggested a switch around the 5-min mark but allowed dyads autonomy to decide for themselves. While we didn't originally set out to analyze engagement, we noted dyads’ light teasing and giggling at each other's interpretations during the sandbox mode.
一名参与者在前臂上戴着触觉显示设备,另一名在触摸屏界面上绘制以传递实时消息。他们在沙盒会话进行到一半时交换了角色;我们在大约 5 分钟的时候轻轻建议交换,但允许双人组自行决定。虽然最初我们并没有打算分析参与度,但我们注意到双人组在沙盒模式期间相互轻松地取笑和咯咯笑。
To evaluate sensitivity, researchers played three of the haptic tracks from the device validation pilot (a triangle, a circle drawn counterclockwise, three dashed lines) for the device wearer; their partner also came up with a few of their own designs. After each haptic track, we asked them to describe the direction, segment count, and overall shape (same parameters selected from the evaluation pilot). We had decided in advance to omit participant designs created by any individual unable to correctly describe two of the three parameters of each of these basic shapes. All participants exceeded this sensitivity threshold.
为了评估灵敏度,研究人员为设备验证试验播放了三个触觉轨迹(一个三角形,一个逆时针画的圆圈,三条虚线)给设备佩戴者;他们的伴侣也提出了一些自己的设计。在每个触觉轨迹之后,我们要求他们描述方向、段数和整体形状(从评估试验中选择的相同参数)。我们事先决定排除任何一个无法正确描述这些基本形状中每个的三个参数中的两个的个体所创建的参与者设计。所有参与者都超过了这个灵敏度阈值。
2) Participant Message Creation
2) 参与者消息创建
To design the haptic encodings, each dyad member was paired with a researcher and led to separate locations to work independently. Researchers helped their participant use the interface to ease the learning curve. Within a session, each participant designed encodings for up to 10 scenarios chosen from Table IV in random order, ending with a wildcard message of their choice. Participants were able to edit, save, and playback their encoding until satisfied before progressing to the next scenario. To ensure that the haptic designs were aligned with the scenario's intended sentiment, researchers read the scenario aloud and asked participants to describe the kinds of feelings that the scenario incited for them before proceeding with the design process.
为设计触觉编码,每个双人组合与一名研究人员配对,并被带到不同的地点独立工作。研究人员帮助他们的参与者使用界面以减轻学习曲线。在一个会话中,每位参与者按照随机顺序从表 IV 中选择的最多 10 个场景设计编码,最后以他们选择的通配符消息结束。参与者可以在满意之前编辑、保存和回放他们的编码,然后再进入下一个场景。为确保触觉设计与场景预期情感一致,研究人员朗读场景并要求参与者描述场景引发的感受种类,然后再继续设计过程。
3) Production of Encodings for Interpretation Phase
3) 为解释阶段生成编码
From this message generation phase, we retained only instances of the haptic encodings where participants verbalized an emotion language that agreed with the sense of the scenario prompt (up to 10 plus a wildcard per participant). A total of 184 unique haptic message encodings were used for the next phase: 167 from the predefined prompts plus 17 wildcards (see Table IV and Fig. 4 for counts and duration respectively).
从这个信息生成阶段开始,我们仅保留了触觉编码的实例,其中参与者口头表达的情绪语言与情境提示的意义一致(每位参与者最多 10 个加一个通配符)。共使用了 184 个独特的触觉信息编码用于下一个阶段:来自预定义提示的 167 个加上 17 个通配符(请参见表 IV 和图 4 ,分别显示计数和持续时间)。
D. Interpretation Phase: In-Person Singles
D. 解释阶段:面对面单身者
To assess how well these haptic encodings could be understood, one week after the Design Study phase we invited all message generating participants back to ‘receive’ a set of messages. Of the original 20, 11 returned from the design phase, including four partner pairs; the other nine were unable to return for the followup due to scheduling constraints so we recruited 10 naive participants.
为了评估这些触觉编码的理解程度,设计研究阶段一周后,我们邀请所有生成消息的参与者回来“接收”一组消息。在最初的 20 名参与者中,有 11 人从设计阶段回来,其中包括四对合作伙伴;另外九人由于时间安排的限制无法参加后续,因此我们招募了 10 名幼稚的参与者。
1) Encoding Set 1) 编码集
Returning participants were given a set of 20 haptic encodings, carefully selected to contain two of each scenario prompt where encodings contained (in random order) messages made by themselves, their partner, or a stranger (on average, 5.2, 5.7, 7.9 messages respectively). For the 10 naive participants, all interpreted encodings that were made by a stranger (
返回的参与者被给予一组 20 个触觉编码,精心挑选包含每种情景提示的两个编码,其中编码包含(随机顺序)由他们自己、他们的伴侣或陌生人制作的信息(平均分别为 5.2、5.7、7.9 条信息)。对于 10 名天真的参与者,他们解释了由陌生人制作的编码(每个
2) Procedure 2) 程序
Message recipients were not able to see the graphical message track at any time during the interpretation phase; they could only feel the sensations. Participants were first asked to freely identify their first impressions of the sensation or what they would instinctively assume the intent to be if they had received the message. They were encouraged to elaborate in a think-aloud format for each haptic message. If unsure, they could replay a messages without limit and/or skip to send it to the back of their queue. After experiencing the message, participants were asked to associate each of 1-very likely; 2-somewhat likely; and 3-very unlikely tags with at most one of the 10 scenarios using a Qualtrics survey application (offered to eliminate the pressure of a single forced choice). They could elect to leave a tag unattached. For interpretation classification, we used the scenario tagged by the highest likelihood (i.e., if 1-very likely was unattached, we use 2-somewhat likely as their top choice). Messages that only get a 3-very unlikely tag was treated as uninterpreted.
接收者在整个解释阶段都无法看到图形消息轨迹;他们只能感受到这些感觉。首先,参与者被要求自由地识别他们对感觉的第一印象,或者如果他们收到消息,他们本能地会认为意图是什么。他们被鼓励以大声思考的方式详细阐述每个触觉消息。如果不确定,他们可以无限次重播消息和/或跳过将其发送到队列的末尾。在体验消息后,参与者被要求使用 Qualtrics 调查应用程序将 1-非常可能;2-有些可能;和 3-非常不可能的标签之一与最多 10 个情景中的一个相关联。他们可以选择不附加标签。对于解释分类,我们使用被标记为最有可能的情景(即,如果 1-非常可能没有被附加,我们将使用 2-有些可能作为他们的首选)。只有获得 3-非常不可能标签的消息被视为未解释。
Finally, all repeat partner-dyad members were asked to interpret the wildcard message that their partner designed specifically for them. Eight (of 11) returned to the study explicitly for this purpose. “I moved an appointment for this! I’m excited to feel what he created for me.” - P04b
最后,要求所有重复合作伙伴成员解释他们的伙伴专门为他们设计的通配符消息。11 名成员中有 8 人明确为此目的返回研究。“我为此改变了一个约会!我很兴奋能感受到他为我创造的东西。” - P04b
Analysis & Results 分析与结果
Our analysis was guided by two primary questions.
我们的分析受到两个主要问题的指导。
(1) Are there feature subsets so characteristic of certain classes of emotion scenario that they may be machine distinguishable? That is, how much of the sentiment can be described by the physical engineering parameters alone?
(1) 情绪场景的某些特征子集是否如此特征明显,以至于机器可以区分出来?也就是说,情感的多少可以仅通过物理工程参数来描述?
We report recognition rates generated using Random Forest (RF), a popular technique for machine learning of affective touch interaction [21], [42], particularly where low data density or strict computation limits preclude sophisticated deep learning models (the former being relevant here).
我们报告使用随机森林(RF)生成的识别率,这是一种流行的情感触摸交互机器学习技术,特别适用于数据稀疏或严格的计算限制下无法使用复杂的深度学习模型(前者在这里是相关的)。
(2) How well, comparatively, can people recognize the sentiment behind the messages if the haptic sensation was designed by (1) a stranger, (2) their partner, or (3) themselves a week ago? For this, we viewed results of our machine classification alongside each of these human-recognition situations.
(2)相对而言,如果触觉感觉是由(1)陌生人、(2)他们的伴侣或(3)自己一周前设计的,人们能够多大程度地识别信息背后的情感?为此,我们观察了我们的机器分类结果以及每种人类识别情况。
Analysis consisted of (Section IV-A) haptic encoding feature extraction and consideration by emotion scenario; (Section IV-B) qualitative examination of the designs themselves, aimed at understanding the diversity of approaches taken; and (Section IV-C) a quantitative look at machine and human interpretation accuracy.
分析包括(第 IV-A 部分)触觉编码特征提取和情感场景考虑;(第 IV-B 部分)对设计本身的定性检查,旨在理解所采取方法的多样性;以及(第 IV-C 部分)对机器和人类解释准确性的定量观察。
A. Features and Parameter Analysis
A. 特征和参数分析
In the following, we describe the data preparation and machine classification process for determining message intent. In overview, we carried out feature extraction for each message, then evaluated feature significance to identify parameters that have the most impact on distinguishing scenarios.
在接下来的内容中,我们将描述用于确定消息意图的数据准备和机器分类过程。总体而言,我们对每条消息进行了特征提取,然后评估了特征的重要性,以识别对区分场景影响最大的参数。
We extracted 82 features (summarized in Table V) from each of the 167 haptic messages. Outside of the user-defined track drawing parameters (diffusion factor and type, brush size, and frequency), we included track characteristics such as the number of discontinuous segments and duration (s) of the entire message (see Fig. 4 for duration distribution by scenario prompt). We also calculated two kinds of displacement: max displacement refers to the euclidean distance between the maximum and minimum coordinates for x and y values of the track; and total displacement refers to the euclidean distance between starting and end points of the drawn track. Max and total velocity use the respective displacement values over the elapsed time. The three Area Under the Curve (AUC) calculations are based on y values by x, x over time, and y over time. We also calculated the full Track Distance as the distance travelled within the display area (excluding any discontinuous jumps); Area Distance is the smallest rectangular area bounded by the track and Area Displacement is the smallest rectangular area bounded by the start and end points of the track. We calculated a set of six statistical functions (min, max, mean, median, variance, and auc) for all remaining vectors: the full set of x and y coordinates across the haptic display as well as the speed=
我们从每个 167 个触觉信息中提取了 82 个特征(总结在表 V 中)。除了用户定义的轨迹绘制参数(扩散因子和类型、画笔大小和频率)外,我们还包括了轨迹特征,如不连续段的数量和整个信息的持续时间(秒)(请参见图 4 ,按场景提示的持续时间分布)。我们还计算了两种位移:最大位移指的是轨迹的 x 和 y 值的最大和最小坐标之间的欧几里得距离;总位移指的是绘制轨迹的起点和终点之间的欧几里得距离。最大和总速度使用相应的位移值除以经过的时间。三个曲线下面积(AUC)计算基于 y 值按 x、x 随时间和 y 随时间。我们还计算了完整的轨迹距离,即在显示区域内行进的距离(不包括任何不连续跳跃);区域距离是由轨迹限定的最小矩形区域;区域位移是由轨迹的起点和终点限定的最小矩形区域。 我们计算了一组六个统计函数(最小值、最大值、平均值、中位数、方差和 AUC)用于所有剩余向量:触觉显示器上所有 x 和 y 坐标的完整集合,以及速度=
To see how physical parameters correlate with the sentiment intent in message generation, we ran a series of ANOVAs on all 82 features. For the features directly controlled by the message designer, specifically Diffusion Factor, Diffusion Type, Brush Size*, Frequency*, Segment Count, Duration, Track Distance*, Area Displacement and Area Distance, the three marked with * were significant at
为了了解物理参数如何与信息生成中的情感意图相关联,我们对所有 82 个特征进行了一系列 ANOVA 分析。对于由信息设计者直接控制的特征,特别是扩散因子、扩散类型、刷子大小*、频率*、段数、持续时间、轨迹距离*、区域位移和区域距离,其中标有*的三个在
B. Qualitative Analysis of Participant Designs
B. 参与者设计的定性分析
Even in the Sandbox (where partners swapped messages face-to-face), distinct approaches emerged for creating track-dependent haptic messages, and continued to develop during the sessions. To capture the diversity of these approaches, we performed a thematic analysis on the 17 designs wherein three raters independently determined 3-6 groupings of the messages. After a lengthy discussion, all raters converged on three high-level categories such that all wildcard designs fall under at least one of (1) direct transcription of some visual representation (either a drawing or writing or other symbology); (2) a rhythmic repetitive sensation that leverages temporal patterns; and (3) distinctive physical sensations that exploit the contrasts between continuous/discontinuous or sharp angular/soft fluttery (see Table VI).
即使在沙盒中(合作伙伴面对面交换信息的地方),为创建依赖于轨道的触觉信息出现了不同的方法,并在会话期间继续发展。为了捕捉这些方法的多样性,我们对 17 种设计进行了主题分析,其中三名评分者独立确定了 3-6 个信息分组。经过长时间的讨论,所有评分者都达成了一致意见,将所有通配符设计归类为至少以下三种高级别类别之一:(1)直接转录某些视觉表现(绘图、书写或其他符号);(2)利用时间模式的节奏重复感觉;以及(3)利用连续/不连续或锐角/柔软颤动之间的对比的独特物理感觉(见表 VI )。
Spanning these categories, we observed approaches that varied both in form of expression (e.g., spatial versus temporal patterns) and in drawing on private shared context, generic references, or abstractions (e.g., literally spelling with letters). These approaches reappear in the unconstrained wildcard messages, which may emulate real world use.
跨越这些类别,我们观察到的方法在表达形式上有所不同(例如,空间与时间模式),并且在利用私人共享背景、通用参考或抽象概念方面也有所不同(例如,字母拼写)。这些方法在无约束的通配符消息中再次出现,这些消息可能模仿现实世界的使用。
We expected spatio-physical sensations designed to evoke interesting haptic experiences. However, we were intrigued to see participants like P10a make pictograms that visually represented the message intent to be traced out on the recipients’ arm in a haptic message (Table VI. Similarly, P05b wrote out a word in Simplified Chinese. We also note the surprising interplay between repetitive discontinuous segments to play with temporal patterns, drawing more on rhythms than spatial representation.
我们期望设计出旨在引发有趣触觉体验的空间物理感觉。然而,我们对于像 P10a 这样的参与者制作象形图表达信息意图,并在触觉信息中被追踪到接收者的手臂上感到好奇(表 VI )。同样,P05b 用简体中文写下一个词。我们还注意到重复的不连续片段之间出现的令人惊讶的相互作用,更多地依赖于节奏而不是空间表现。
C. Interpretation Accuracy
C. 解释准确性
When people touch one another, a plethora of social cues and emotional content can be conveyed, particularly between intimate partners [43]; considerable affective content is also communicated through touch between strangers [13]. We wonder if social content communicated in close relationships can be sent and interpreted more accurately compared to that between strangers, particularly when we add a machine interlocutor. For insight, we first compared recognition accuracy by machine and human strangers; neither of these have personalized training nor shared history with message designers.
当人们彼此触摸时,尤其是在亲密伴侣之间,可以传达大量社交暗示和情感内容;陌生人之间的触摸也传达了相当多的情感内容。我们想知道在亲密关系中传达的社交内容是否可以与陌生人之间的传达相比更准确地发送和解释,尤其是当我们加入一个机器对话者时。为了洞察,我们首先比较了机器和人类陌生人的识别准确性;这两者都没有个性化培训,也没有与信息设计者共享历史。
Fig. 6 uses accuracy confusion matrices to compare classification outcomes for four cases of interest: by machine, human stranger, partner, and self. Correctly classified instances are on the diagonal.
图 6 使用准确性混淆矩阵来比较四种感兴趣情况的分类结果:由机器、陌生人、伴侣和自己。正确分类的实例位于对角线上。
By Machine Stranger:
We consider the use case where a machine interprets the message and communicates a best guess to the intended recipient. For this to work, we conceive of a procedure where a model is trained on the haptic encodings labelled with the emotion ascribed to the presented scenario. We selected Random Forest (RF) as our classifier, as the literature has shown RF to work well with affective and social touch [21], [42], [44], [45], [46]. We found (Fig. 6(b)) that 10-fold cross validation using a subject-dependent (touches from same participant may be in both training and test data) RF classifier on 167 messages achieved an overall accuracy of 18.6% (chance 10%).
通过机器陌生人:我们考虑一个使用情况,即机器解释消息并向预期接收者传达最佳猜测。为了使其正常工作,我们构想了一个程序,其中一个模型在以情感标记的触觉编码上进行训练,以描述所呈现的情景。我们选择随机森林(RF)作为我们的分类器,因为文献表明 RF 在情感和社交触摸方面表现良好。我们发现(图 6(b) ),使用一个基于主体的十折交叉验证(来自同一参与者的触摸可能同时出现在训练和测试数据中)RF 分类器对 167 条消息进行分类,整体准确率达到 18.6%(机会准确率为 10%)。
By Human Stranger:
Machine classification and stranger interpretation are both performed on messages by unknown designers and there is little or no shared history, so it is interesting to compare these results. When we asked participants to evaluate designs by strangers, out of 274 trials (in which some of the 167 encodings are repeated), people's best guess – the prompt they thought was the most likely match – was accurate 17.9% (chance 10%) of the time, compared to 18.6% by the RF classifier.
通过人类陌生人:机器分类和陌生人解释都是在未知设计者的消息上执行的,并且几乎没有共享历史,因此比较这些结果是很有趣的。当我们要求参与者评估陌生人设计时,在 274 次试验中(其中有 167 个编码是重复的),人们最佳猜测 - 他们认为最可能匹配的提示 - 在 17.9%的时间内准确(机会为 10%),而 RF 分类器为 18.6%。
By Relationship.
We can look at recognition rate of the intended emotion in each message prompt to see how sentiment communication varies depending on the relationship (through the confusion matrices of Fig. 6). Here we see that overall, message designers recognize their own messages most often (Fig. 6(d), 31.6%) and that of strangers (Fig. 6(a), 17.9%) the least, with partners in between (Fig. 6(b), 22.2%).
通过关系。我们可以查看每个消息提示中预期情绪的识别率,以了解情感沟通如何因关系而异(通过图 6 的混淆矩阵)。在这里,我们看到总体上,消息设计者最常识别自己的消息(图 6(d) ,31.6%),陌生人的消息最少(图 6(a) ,17.9%),而伙伴之间居中(图 6(b) ,22.2%)。
However, the story becomes more complicated: some prompts defy the expectation that interpretation accuracy increases with relationship closeness. We summarize message interpretation accuracy by relationship in Fig. 7. Interestingly, anger, miss, and sad are consistently poorly recognized by the designers themselves.
然而,故事变得更加复杂:一些提示违背了解释准确性随关系亲近程度增加的预期。我们通过图 7 总结了关系中的信息解释准确性。有趣的是,愤怒、思念和悲伤在设计师自己中一直被辨识得很差。
Of Wildcard Messages:
Of the 20 design phase participants, 11 returned for interpretation. We played each one the wildcard message that their partner designed specifically for them. These messages were completely open-ended (presumably very low chance of randomly guessing correctly).
关于通配符消息:在 20 名设计阶段参与者中,有 11 人返回进行解释。我们向每个人播放了由他们的伙伴专门为他们设计的通配符消息。这些消息是完全开放式的(假定随机猜测正确的机会非常低)。
For wildcards, we marked an interpretation Correct when two independent scorers agreed that the interpretation matched the intent. Scorers looked for word matches, synonyms, and common sentiments. Interestingly, 7 of 11 message recipients were able to correctly interpret the message (summarized in Table VII), a recognition rate of 63.6%, higher than most other emotion prompts.
对于通配符,当两位独立的评分者一致认为解释符合意图时,我们标记为正确的解释。评分者寻找单词匹配、同义词和共同情感。有趣的是,11 位消息接收者中有 7 位能够正确解释消息(总结在表 VII 中),识别率为 63.6%,高于大多数其他情绪提示。
Some interesting interpretations include P07a's design jigsaw puzzle pieces (Fig. 1(d)) to communicate that they ”fit together”. Partner P07b recognized the message as representing “connection... like a jigsaw puzzle” (Table VII).
一些有趣的解释包括 P07a 设计的拼图块(图 1(d) ),以传达它们“相互契合”。合作伙伴 P07b 认识到这个信息代表着“连接...就像拼图”(表 VII )。
A more abstract design is P06b's signal where two lines followed by a lengthy swirl communicates an “eyeroll” about “... a person who annoys [my partner and me]”(Fig. 1(e)). Upon feeling the encoding, partner P06a immediately recognized it as communicating irritation since the rhythmic pattern was reminiscent of “... the pacing [P06b would] use to say ’oh. my. GAWWD.’ with” - P06a.
P06b 的信号设计更抽象,其中两条线后跟着一个漫长的涡旋,传达了关于“…一个让[我的伴侣和我]烦恼的人”的“翻白眼”的情绪(图 1(e) )。在感受到这种编码后,伴侣 P06a 立即意识到这是在传达烦躁情绪,因为这种节奏模式让人想起“…P06b 会用来说‘哦。我的。天。’的步调” - P06a。
Discussion 讨论
Our primary study goal was to learn how to more effectively leverage social touch in haptic messaging. While our results generally corroborate the literature asserting that contextual background and shared history play an important role in the perception of emotional content [12], [14], [34], we now return to our research questions and discuss how our data provides evidence toward answering them.
我们的主要研究目标是学习如何更有效地利用触觉信息中的社交触摸。尽管我们的结果通常证实了文献中关于背景语境和共享历史在情感内容感知中起重要作用的观点,但现在我们将回到我们的研究问题,并讨论我们的数据如何提供证据来回答这些问题。
A. Message Design Observations
A. 信息设计观察
Our study's encoding designers were tasked with communicating rather complex social meanings. They were given only a short time to learn an unfamiliar device limited to sensations that are low-resolution and unnatural relative to direct human touch, albeit shown to encode interpretable affective content [15], [27], [47]. Furthermore, participants designed with researcher assistance which may introduce other biases – e.g., participants may accept researcher suggestions more readily than if they were alone. Departures from the ideal use aside, we wish to know more about the haptic message design experience: how do people approach haptic message design given our current prototype and scenario prompts?
我们研究中的编码设计师被要求传达相当复杂的社会意义。他们只有很短的时间来学习一种陌生的设备,该设备受限于低分辨率和与直接人类触摸相比较不自然的感觉,尽管已被证明可以编码可解释的情感内容。此外,参与者在研究人员的协助下设计,这可能引入其他偏见 - 例如,参与者可能更容易接受研究人员的建议,而不是独自一人时。除了远离理想使用之外,我们希望了解更多关于触觉信息设计体验:人们如何在我们当前的原型和情境提示下进行触觉信息设计?
Do Design Strategies Reveal Shared History?
By examining the wildcard message designs, we suspect shared history is embedded in the visual drawing strategy and the rhythmic/temporal patterns, particularly evident in P06a recognizing her partner's idiosyncratic speaking cadence (Fig. 1(e)).
设计策略是否揭示了共享历史?通过检查通配符消息设计,我们怀疑共享历史嵌入在视觉绘图策略和节奏/时间模式中,特别是在 P06a 中,她认识到伴侣独特的说话节奏(图 1(e) )。
The shape-drawing strategies also evoke common backgrounds, illustrated by P5b's communication in their native written language, who shared that “this is like a game that my parents used to play with me as a child. They would write a character on my arm or back and ask me to guess what they wrote”. P07b's design based on connecting puzzle pieces (Fig. 1(d) – interpreted accurately despite its complexity) made us wonder if this pair might enjoy doing jigsaw puzzles together.
形状绘图策略还唤起了共同的背景,如 P5b 用他们的母语书写进行交流,他们分享说:“这就像我父母小时候和我玩的一个游戏。他们会在我的手臂或背上写一个字,然后让我猜他们写的是什么。”基于连接拼图块的 P07b 的设计(图 1(d) - 尽管复杂但被准确解释)让我们想知道这对是否喜欢一起做拼图游戏。
While acknowledging that the touchscreen interface may have suggested a visual approach (drawing and writing) we are encouraged by the range of content that screen-sketching supports. Partners appear to draw recognizable patterns from multiple senses – most notably visually and vocally (as in Fig. 1(e)) – raising questions about how these and other approaches might evolve with more time.
尽管承认触摸屏界面可能暗示了一种视觉方法(绘画和书写),但我们对屏幕素描支持的内容范围感到鼓舞。合作伙伴似乎能够从多种感官中绘制出可识别的模式 - 尤其是视觉和声音方面(如图 1(e) 所示) - 这引发了关于这些和其他方法如何随着时间的推移而发展的问题。
Are there Universal Messages?
Designers played with physical parameters like frequency and brush diameter to determine whether the sensation communicated the right intent. Fig. 5 highlights variation in the interaction between significant design parameters, creating a parameter “footprint” by message. Designers seemed to match lower arousal [48] message intentions (like calm, miss, sad) using small brush diameter and low frequency.
是否存在普遍信息?设计师们通过调整频率和刷子直径等物理参数来确定感觉是否传达了正确的意图。图 5 突出了重要设计参数之间的交互变化,通过信息创建了一个参数“足迹”。设计师们似乎通过使用小刷子直径和低频率来匹配较低激励 [48] 信息意图(如平静、思念、悲伤)。
Interestingly, attention was often designed with the largest brush at the highest frequencies, but also had the largest footprint – the greatest design variation across all designed messages. This suggests that there are other factors attributable to highly human-interpretable messages which transcend design consistency: i.e., some concepts might be broadly amenable to many representations, or they might be extremely personal and our participants were able to find the particular encoding that worked for their partner.
有趣的是,关注通常是在最高频率下设计的,但也具有最大的影响范围 - 在所有设计信息中具有最大的设计变化。这表明高度易于人类解释的信息可能存在其他因素,超越设计一致性:即,一些概念可能适用于许多表达方式,或者它们可能是极其个人化的,而我们的参与者能够找到适合他们伙伴的特定编码。
B. Interpretation Rate B. 解释率
We preface discussion of accuracy by noting that while we carefully built on lessons from past research, due to divergence of our evaluation objectives in this novel application space, differences in our approach preclude directly comparing recognition accuracy. For example, there is an overlap of only four emotion prompts with [13], where the methodology is very different (direct contact rather than machine-mediated touch). Even where results are similar when comparing with other device-mediated affective touch ([36] also presents accuracy that roughly doubles chance recognition), these works have very distinct types of touch constraints, instructions, and evaluation methods, again discouraging direct comparison. In all cases, mapping vibrotactile sensations to emotive thoughts can be non-intuitive, and in many ways it is remarkable that interpretation accuracy would ever exceed chance.
我们在讨论准确性之前,要注意虽然我们仔细借鉴了过去研究的经验教训,但由于我们在这个新颖应用领域的评估目标存在差异,我们的方法也有所不同,因此无法直接比较识别准确性。例如,与 [13] 仅有四个情绪提示重叠,而方法论却大相径庭(直接接触而非机器介导触摸)。即使与其他设备介导的情感触摸进行比较时结果相似( [36] 也呈现出准确性大约是随机识别的两倍),这些作品具有非常不同类型的触摸约束、指令和评估方法,再次阻碍了直接比较。在所有情况下,将振动触觉感觉映射到情感思维可能是不直观的,而且在许多方面,令人惊讶的是解释准确性竟然会超过随机猜测。
Our work and others’ demonstrate that digitally mediated affective communication is feasible. However, it is important to keep in mind that these relatively low instance counts (particularly for the self and partner interpretation sets as seen in Fig. 6) provide only limited insight into message efficacy. Thus, our primary benchmark of comparison for this exploration is what people themselves can do in the same study conditions.
我们的工作和其他人的工作表明,数字化中介的情感沟通是可行的。然而,重要的是要记住,这些相对较低的实例计数(特别是对自己和伴侣解释集,如图 6 所示)仅提供了有限的关于消息效果的见解。因此,我们对这一探索的主要比较基准是人们在相同研究条件下能做到什么。
Why do Designers not Recognize Their Own Designs?
Generally, relationship closeness does influence interpretation accuracy of social touch ([43] and here, Fig. 7). Thus we would expect designers to be best at recognizing the messages they had designed. However, this is not always the case. Fig. 7 highlights how message prompts for anger, miss and sadness or sorry seem to be recognized more accurately by partners and strangers than by their designers. While this certainly needs more investigation, we observe that touch behaviours communicating these sentiments are especially likely to be directed to another – e.g., one is unlikely to miss, or show longing for, oneself. Possibly, this leads to our being less likely to recognize our own touch when expressing sentiments with this quality.
为什么设计师无法认出自己的设计?一般来说,亲密关系确实会影响社交触摸的解释准确性( [43] ,见图 7 )。因此,我们本应期望设计师最擅长认出他们设计的信息。然而,情况并非总是如此。图 7 突显了愤怒、错过、悲伤或抱歉等信息提示似乎被伴侣和陌生人比设计师更准确地认出。虽然这当然需要更多的调查,但我们观察到传达这些情感的触摸行为特别可能是针对他人的 - 例如,一个人不太可能错过自己,或者向自己表达渴望。可能,这导致我们在表达具有这种特质的情感时更不太可能认出自己的触摸。
How Can Wildcard Recognition be so High?
While most messages have interpretation accuracy under 35%3 (chance 10%), wildcard messages – with no specified prompt and thus, no fixed interpretation option – is recognized surprisingly well at 7 correct interpretations out of 11 messages (Table VII). The wildcard messages may be the best examples of closeness in relationship improving interpretation of message intent. People with a shared history can draw from a wealth of experiences to generate creative messages, even idiosyncrasies from other modalities. Speech rhythm and cadence (P06a) is one example, but we can imagine haptic sensations that emulate an impatient tapping foot or short strokes that channel dumbfounded cartoon blinks.
通配符识别为何如此高?虽然大多数消息的解释准确率低于 35%(机会 10%),但通配符消息——没有指定提示,因此没有固定的解释选项——在 11 条消息中以 7 个正确解释的惊人表现被认可(表 1)。通配符消息可能是关系亲密度提高消息意图解释的最佳例证。有共同历史的人可以借鉴丰富的经验产生创造性消息,甚至可以从其他模态性中产生怪癖。语音节奏和韵律(P06a)就是一个例子,但我们可以想象模拟急躁的脚踢或模仿茫然的卡通眨眼的触觉感觉。
Shared recent context surely impacts message interpretation rate. If P09a and P09b had been fighting on the way to the study session about one eating more than their share of dinner, then shrimp may have been on both of their minds (Table VII). A recent charged memory could make a highly specific message easy to read, maybe even irrespective of the haptic design. Removing the shared history by getting strangers to interpret the general content wildcard messages (excepting disagreement over shrimp) could be illuminating.
最近共享的背景肯定会影响信息解读速率。如果 P09a 和 P09b 在去学习会议的路上因为其中一人吃了比自己份额更多的晚餐而争吵,那么虾可能会在他们两人的脑海中(表 VII )。最近的充满情感的记忆可能会使高度具体的信息易于阅读,甚至可能不受触觉设计的影响。通过让陌生人解释一般内容的通配符信息(除了对虾的争议),去除共享历史可能会有启发性。
What is the Potential for Machine Recognition?
Machine recognition rates were comparable to stranger recognition (18.6% and 17.9% respectively, chance 10%). Comparing Fig. 6(a) and (c)'s confusion matrix diagonals shows that messages of anxious, gratitude, miss and sad are better recognized by machines than human strangers. Fig. 5 shows distinct patterns of common design parameters, particularly for anxious (small active area with small brush diameter and low track length across a large vibration frequency range) and gratitude (similarly small active area but with vibration frequency mostly in the low end). Perhaps these ranges are statistically distinct but tactually imperceptible, making it more difficult for human interpreters.
机器识别的潜力是什么?机器识别率与陌生人识别率相当(分别为 18.6%和 17.9%,机会为 10%)。比较图 6(a) 和 (c) 的混淆矩阵对角线显示,焦虑、感激、思念和悲伤等信息被机器比人类陌生人更好地识别。图 5 显示了常见设计参数的明显模式,尤其是焦虑(小活动区域、小刷直径和低跟踪长度跨越大振动频率范围)和感激(同样小的活动区域,但振动频率主要在低端)。也许这些范围在统计上是不同的,但在触觉上是难以察觉的,这使得人类解释者更难理解。
Because message designers created only one design for each message prompt, we have a sparse training set with no repetition on the interaction of two important dimensions (designer and message). Affective touch interaction is individual, so machine recognition increases dramatically with more person-specific training [21]. We see an opportunity for additional training samples to complement shared contextual history where machine recognition may serve to support interpersonal message interpretation accuracy.
由于信息设计师为每个信息提示仅创建了一个设计,我们拥有一个稀疏的训练集,在两个重要维度(设计师和信息)的交互上没有重复。情感触摸交互是个体化的,因此随着更多针对个人的训练,机器识别显著增加。我们看到一个机会,即通过额外的训练样本来补充共享的背景历史,其中机器识别可能有助于支持人际信息解释的准确性。
C. The Messaging Experience
C. 信息传递体验
Despite not setting out to evaluate the ‘fun factor’ of the messaging experience on our haptic animation prototype, we discovered that the design sessions where pairs worked together (sandbox mode) were often punctuated with giggles and gentle ribbing (“What? You mean you can't feel that's a heart?!” – P02b to P02a) as close friends and partners were first developing a sense for how to use the device. We noted that at least 11 of 20 participants spoke about playfulness, happiness, and/or laughter while designing their wildcard messages, revealing extra pleasure in imagining their partners puzzling out meanings involving private context and some amount of effort. The fun generally emerged through affectionate collaboration – it was not a solo activity. Here, we discuss valuable observations of the design-interpretation process and speculate about improvements necessary before it can become a viable communication channel.
尽管我们并没有旨在评估我们的触觉动画原型消息体验的“乐趣因素”,但我们发现,在配对共同工作的设计会话中(沙盒模式),经常会夹杂着咯咯笑声和轻松的取笑(“什么?你是说你感觉不到那是一颗心吗?!”- P02b 对 P02a),因为亲密的朋友和伴侣们首次开发了如何使用设备的感觉。我们注意到,在 20 名参与者中至少有 11 人在设计他们的通配消息时谈到了愉快、幸福和/或笑声,揭示了在想象他们的伴侣费解涉及私人背景和一定努力的含义时额外的愉悦。乐趣通常是通过亲切的协作而产生的-这不是一个单独的活动。在这里,我们讨论了设计-解释过程的宝贵观察,并推测在它成为可行的沟通渠道之前需要进行的改进。
How did Individuals Vary Across the Pipeline of Interpretation?
We expect that successful haptic messaging likely depends on both the subjective tactile perception and interpretation of the message intent; each of these are themselves complex processes. The first depends on display performance (nature of stimuli, resolution, dynamic range, etc), skill of the designer's use of it, and the individual perceptual sensitivity of the recipient. The second is where we were able to focus more in the present study, looking at factors like relationship and message type.
个体在解释的过程中如何变化?我们预计,成功的触觉信息传递可能取决于主观触觉感知和信息意图的解释;这两者本身都是复杂的过程。第一个取决于显示性能(刺激的性质、分辨率、动态范围等)、设计者使用技巧以及接收者的感知敏感度。第二个是我们在本研究中更多关注的地方,关注因素如关系和信息类型。
Our study protocols included checking on comfort and threshold proficiency in using the system, but did not comprehensively measure individual acuity or its components or demographic influences. Thus we cannot speak to the degree to which perceptual challenges (as well as stimulus type suitability – i.e., of vibrotactile modality for affective messages) impacted individuals and pairs’ ability to use the system and enjoy the interactions.
我们的研究方案包括检查系统的舒适度和阈值熟练度,但并未全面衡量个体敏锐度或其组成部分或人口统计学影响。因此,我们无法确定感知挑战(以及刺激类型的适用性 - 即,对情感信息的振动触觉模式)对个体和配对使用系统和享受互动的能力产生了何种影响。
We informally observed a wide range of skill in both individuals’ and dyads’ ability to construct or fully carry out a communication chain – typical for the studies involving either tactile acuity or emotional intelligence. Acuity arises both from sensitivity to ranges of sensation, and at a higher level, the ability to mentally integrate then identify shapes that are received as spatio-temporal line drawings on the skin. For example, we were particularly impressed by the recipient of the wildcard jigsaw puzzle pieces: these pictographs are complex with many vertices and two separate but closely set, compatibly interlocking components. This integrative feat seems remarkable, and likely beyond the capability of most other participants or indeed the researchers. However, it is a fascinating example of what might be possible, and may have been aided by contextual factors that improved this recipient's guessing odds.
我们非正式观察到个体和二人组在构建或完全执行沟通链条的能力方面存在广泛的技能差异 - 这在涉及触觉敏锐度或情绪智力的研究中很典型。敏锐度既来自对感觉范围的敏感性,也来自在更高层次上,能够在皮肤上接收到的时空线条图形上进行精神整合然后识别形状的能力。例如,我们特别对万能拼图游戏碎片的接收者印象深刻:这些象形文字非常复杂,具有许多顶点和两个分开但紧密设置的、相互契合的组件。这种整合的壮举似乎很了不起,很可能超出了大多数其他参与者甚至研究人员的能力。然而,这是一个令人着迷的例子,展示了可能性,并且可能受到改善这位接收者猜测几率的情境因素的帮助。
What is the Longitudinal Prognosis?
These were one-shot design efforts. People can learn to adjust to a partner and to a communication medium. We wonder how individual and dyad performance would improve over time, and how pairs might evolve and enrich their communication style – what strategies they would come to rely on or discard; how memory would work, stability of vocabulary (already anticipated by P02b – Fig. 9), what kinds of context (short or long term) they would leverage when given the opportunity. We wonder if the interaction would become more engaging and/or valuable as a core communication modality when more familiar, or soon set aside. Our present results are a promising start, but real answers await longer studies and a device and editor that could function in everyday life.
纵向预后是什么?这些是一次性的设计努力。人们可以学会适应伴侣和沟通媒介。我们想知道个人和双人表现会随着时间的推移而改善,以及双方可能如何发展和丰富他们的沟通风格 - 他们会依赖或放弃哪些策略;记忆如何运作,词汇的稳定性(已被 P02b 预期 - 图 9 ),他们在有机会时会利用哪种类型的语境(短期或长期)。我们想知道如果互动变得更具吸引力和/或更有价值,作为更熟悉的核心沟通方式,或者很快被搁置。我们目前的结果是一个有希望的开始,但真正的答案等待更长时间的研究以及一个可以在日常生活中发挥作用的设备和编辑器。
How Could we Improve Our Prototype?
Inherent to the messaging experience is the device and interface. We built a minimally viable prototype to establish the feasibility of affect-content communication via a wearable haptic animation display. Our findings in design variation, interpretation rate, and overall enthusiastic reception suggest that even our simple, low-resolution prototype can open up a rich and evocative haptic playground. To further enrich the experience, subsequent iterations of the hardware could integrate smaller, more powerful tactors to increase end effector density (i.e., allow for more tactors to fit in the same surface area), which may afford more intricate designs. The most apparent example of spatial resolution or sensitivity discrepancy was evident during the Sandbox phase with people like P02b incredulous with her partner's (P02a) inability to recognize the more intricate shapes. ”It's clearly got angles though babe!” – P02b when P02a mis-identified an octagon as a circle.
我们如何改进我们的原型?消息体验的固有特性是设备和界面。我们建立了一个最小可行原型,以建立通过可穿戴触觉动画显示进行情感内容交流的可行性。我们在设计变化、解释率和整体热情接受方面的发现表明,即使是我们简单的、低分辨率的原型也可以打开一个丰富而引人入胜的触觉游乐场。为了进一步丰富体验,硬件的后续迭代可以集成更小、更强大的触觉器件,以增加末端效应器密度(即允许更多触觉器件适应相同的表面积),这可能会带来更复杂的设计。在沙盒阶段,最明显的空间分辨率或灵敏度差异的例子是在像 P02b 这样的人与她的伙伴(P02a)在识别更复杂的形状时无法认出的情况。“明显是有角度的,亲爱的!”——P02b 当 P02a 将一个八边形误认为圆形时。
The design interface could also be amended with more fine-grained control mechanisms. Although only one design participant out of 20 asked, we can envision a scenario where experienced users may want to design messages with time-varying frequency and amplitude, dramatically increasing the range and complexity of the design space.
设计界面也可以通过更精细的控制机制进行修改。虽然只有 20 名设计参与者中的一名提出了这个问题,但我们可以设想一种情景,即有经验的用户可能希望设计具有时间变化频率和幅度的消息,从而显著增加设计空间的范围和复杂性。
Corroborating findings from the hand-based Tactile Emoticon (featuring haptic sensations of temperature, vibration, and pressure) [25], we posit that so long as users are provided a sufficient customization range for designs and design strategies, partners may play around to come up with something that works for them, regardless of device sophistication. We expect iterations of device and messaging application to inform one another; here, we present a promising proof of concept as a strong starting point.
从基于手的触觉表情(具有温度、振动和压力的触觉感觉)的结果得出的结论 [25] ,我们认为只要用户为设计和设计策略提供了足够的定制范围,合作伙伴可以尝试不同的方法,找到适合他们的东西,而不受设备复杂性的影响。我们期望设备和消息应用的迭代相互影响;在这里,我们提出一个有前景的概念验证作为一个强有力的起点。
Conclusion 结论
We presented a multi-phase study on machine-mediated social touch to shed light on how people might create and interpret emotion-encoded haptic messages. We used a custom wearable spatial tactile display, and an interface for participants to compose spatiotemporal patterns. The study's scope included scenario prompt sourcing, message encoding design, and message interpretation. Its design and analysis highlight the influence of relationship and shared context on how communication plays out. We summarize the key findings sparking future lines of inquiry.
我们展示了一项关于机器介导的社交触觉的多阶段研究,以阐明人们如何创造和解释以情感编码的触觉信息。我们使用了定制的可穿戴空间触觉显示器,以及一个供参与者编写时空模式的界面。该研究的范围包括情景提示的获取、信息编码设计和信息解释。其设计和分析突出了关系和共享背景对沟通展开方式的影响。我们总结了激发未来研究方向的关键发现。
1. A shared history between message designer and interpreter generally improves message comprehension: private inside jokes are a great strategy; individuals are not always great at reading their own tactile-writing.
Overall, message interpretation accuracy increases from strangers (17.9%) to partners (22.2%) to message designers themselves after a week (31.6%). However, partners could understand 7/ 11 of open-ended wildcard messages, a surprisingly high accuracy given their unconstrained content. We posit that shared contextual knowledge is of great value; and further note that the sharing was almost always both humorous and private in nature – couples sharing a private inside joke, with their common experience the key that unlocked understanding.
1. 消息设计者和解释者之间的共享历史通常会提高消息理解:私人内部笑话是一个很好的策略;个人并不总是擅长阅读自己的触觉书写。总体而言,消息解释的准确性从陌生人(17.9%)到伴侣(22.2%)再到一周后的消息设计者自身(31.6%)都有所增加。然而,伴侣可以理解 7/11 的开放式通配符消息,这种准确性出乎意料地高,考虑到它们的内容是不受限制的。我们认为共享的背景知识非常有价值;并进一步指出,分享几乎总是既幽默又私人化的性质 - 情侣分享私人内部笑话,他们共同的经历是解锁理解的关键。
We also found that some messages were poorly recognized by designers themselves compared to their partners. We speculate that physical manifestations of anger and missing or longing are not often directed at ourselves, so we are less likely to recognize our own – but need more than one message per designer to be sure.
我们还发现,与他们的伙伴相比,一些信息被设计师自己认识地不够好。我们推测,愤怒和缺失或渴望的身体表现通常不是针对自己的,因此我们更不太可能认出自己的情绪 - 但需要更多的信息来确认每位设计师。
2. Machines are about as good as strangers at haptic message interpretation (for now).
Our machine classifier recognized message intent with 18.6% accuracy, comparable to that of strangers (chance 10% in all cases) where closer relationships between sender and receiver serve to improve interpretation rates overall. We imagine that insofar as machine-‘strangers’ and human-strangers lack shared context, both relationships are similarly distant with the message sender. Future work could inspect whether personalized training may offer a machine analogue for ‘history’.
2. 机器在触觉信息解释方面与陌生人差不多(目前)。我们的机器分类器以 18.6%的准确率识别信息意图,与陌生人的准确率相当(所有情况下的机会为 10%),发件人和收件人之间更亲近的关系有助于提高整体解释率。我们认为,就机器“陌生人”和人类“陌生人”缺乏共享背景而言,两种关系与信息发送者同样疏远。未来的工作可以检查个性化培训是否可以为“历史”提供机器类比。
3. Individual design strategies may co-opt other modalities.
Some designers produced visually recognizable pictograms to communicate message sentiment – puzzle pieces and happy faces drawn on the touchscreen – while others played with rhythm – discrete taps simulating excited poking behaviour, or strokes and spirals timed to mimic an idiosyncratic speaking cadence. Given this early diversity, how might design behaviour mature if pairs had more time to trade messages?
3. 个体设计策略可能会利用其他模态。一些设计师制作了在触摸屏上绘制的视觉可识别的象形文字来传达信息情感 - 如拼图块和快乐的笑脸 - 而其他人则玩弄节奏 - 离散的轻拍模拟兴奋的戳戳行为,或者根据特定的说话节奏来计时的笔画和螺旋。鉴于这种早期的多样性,如果配对有更多时间交换信息,设计行为会如何发展?
Next Steps.
This study has benchmarked interpretability rates and highlighted encoding strategies for a relatively expressive haptic display (relying on spatiotemporal animation, supporting drawing-type designs). Looking ahead, our findings underscore the importance of considering design strategies when choosing displays and editing systems, that maximize expressive capability; and that dyad communication is highly unique, rich with many characteristics helpful in maintaining emotional connection in relationships.
下一步。本研究已经对可解释性率进行了基准测试,并突出了相对表达丰富的触觉显示的编码策略(依赖于时空动画,支持绘图类型设计)。展望未来,我们的发现强调了在选择显示和编辑系统时考虑设计策略的重要性,以最大限度地发挥表现能力;并且双人沟通是非常独特的,富有许多特征,有助于在关系中保持情感联系。
Obvious next steps are to develop physical displays that are practical and comfortable in real settings yet at least as expressive as the one used here – and are fun to use. Then it will be possible to launch studies that monitor how vocabulary used by dyads develops and enriches or withers over time, and the contribution this kind of communication makes to pairs who cannot be together. So when is a haptic message like an inside joke? We think it matters only when there's someone you care to share it with.
明显的下一步是开发在实际环境中既实用又舒适,至少与此处使用的显示器一样富有表现力且使用起来有趣的物理显示器。然后就可以启动研究,监测双人组使用的词汇如何随着时间的推移发展和丰富,或者逐渐消失,以及这种沟通方式对无法在一起的双方的贡献。那么,触觉信息何时像一个内部笑话?我们认为,只有当有人愿意与之分享时,这才重要。
ACKNOWLEDGMENTS 致谢
This work was supported by Reality Labs Research, Meta Platforms Inc. and conducted as part of a research internship. We are grateful to the many people who have supported the production of this work: Taylor Bundy and Casey Brown with data collection; Blaise Ritchie with the prototype's design interface; Rubia Guerra and Hannah Elbaggari with figures and reporting analysis; and many others for their supportive input, constructive comments, and helpful edits.
本工作得到了 Reality Labs Research、Meta Platforms Inc.的支持,并作为研究实习的一部分进行。我们要感谢许多人对本工作的支持:Taylor Bundy 和 Casey Brown 负责数据收集;Blaise Ritchie 负责原型设计界面;Rubia Guerra 和 Hannah Elbaggari 负责图表和报告分析;还有许多其他人提供了支持性意见、建设性评论和有益的修改。