1Supplementary Materials Experiment 1

1Supplementary Materials Experiment 1

1.1MANOVA Manipulation checks

All the participants’ ratings on the manipulation check variables (gaze, fluency, hesitate, and mistakes) were subjected to a 2 (gaze behavior: no look vs. look) × 2 (motion fluency: smooth vs. shake) × 2 (hesitation: no hesitate vs. hesitate) × 2 (task performance: no mistakes vs. mistakes) between subject MANOVA. This analysis yielded significant multivariate effects on the manipulations of the robot’s gaze, motion fluency, hesitation behavior, and mistakes, as well as the fluency × hesitate interaction effect (Table 1).

Effect / Wilks’ λ / F(4, 137) / p
gaze / 0.90 / 4 / .004
fluency / 0.26 / 95.82 / < .001
hesitate / 0.72 / 13.15 / < .001
mistakes / 0.27 / 90.68 / < .001
fluency×hesitate / 0.92 / 3.05 / 0.02

Table 1: Significant results of the MANOVA on the manipulation checks.

For the individual effects of the manipulation checks, univariate ANOVAs were conducted. These results, summarized in Table 2, show that all manipulation checks were significantly affected by their intended manipulations, and they are all the predicted directions. This indicates that participants noticed all manipulations of the robot’s behavior.

The manipulation checks of gaze, motion fluency, and hesitation behavior were also significantly affected by other manipulations. The mistakes made by the robot had a main effect on the manipulation check of the gaze behavior, possibly indicating that the participants attributed that the mistakes made by the robot to a failing looking behavior. The motion fluency manipulation had a significant effect on the manipulation check of hesitation behavior and vice versa; indicating that motion fluency are interpreted as hesitations and hesitations as trembling behavior. There was also a significant motion fluency × hesitate interaction effect on the fluency manipulation check. Post-hoc t-tests, summarized in Table 3, show that the effect of hesitation behavior is only present when the robot is not trembling (see also Figure 1).

Manipulation / effect / F(1,140) / p / ηp2 / level: off / level: on
check / M / SD / M / SD
gaze / gaze / 14.46 / .001 / .09 / 4.73 / 1.66 / 5.61 / 1.47
mistakes / 43.91 / .001 / .24 / 5.89 / 1.22 / 4.41 / 1.66
fluency / fluency / 369.48 / .001 / .73 / 2.65 / 1.54 / 6.22 / 0.86
hesitate / 20.85 / .001 / .13 / 4.04 / 2.32 / 4.84 / 1.95
fluency × hesitate / 11.00 / .001 / .07 / seeTable 3
hesitate / hesitate / 41.05 / .001 / .23 / 3.1 / 1.73 / 4.77 / 1.64
fluency / 5.05 / .026 / .03 / 4.04 / 2.32 / 4.84 / 1.95
performance / mistakes / 342.27 / .001 / .71 / 6.09 / 1.18 / 2.61 / 1.17

Table 2: Descriptives of significant effects (p < .006) on the manipulation checks Van Halen Video study. All factors were significant on their respective control questions. Of note is the effect of the hesitation behavior and the motion fluency × hesitation behavior on motion fluency, which is further explored in Table 2.

Hesitate
t(76) / p / r / No Hesitate / Hesitate
Motion fluency / M / SD / M / SD
Fluent / -4.76 / < .001 / .23 / 1.92 / 1.01 / 3.38 / 1.63
Shake / -1.24 / .22 (n.s.) / n.s

Table 3: Post hoc t-tests of the motion fluency × hesitate interaction on fluency.In the event when the robot is fluent, there is a significant difference whether the robot does not hesitate (M = 1.92, SD = 1.01) or does hesitate (M = 3.38, SD = 1.63), t(76) = −4.76, p < .001. This difference is not significant when the robot is not fluent, t(76) = −1.24, p = .215.

Figure 1: Bar plot of the hesitate ×motion fluency interaction effect on the motion fluency manipulation check. Error bars represent the 95% confidence interval.

1.2Results exploratory measures

Autonomy

A univariate 2 (gaze behavior: no look vs. look) × 2 (motion fluency: smooth vs. shake) × 2 (hesitation: no hesitate vs. hesitate) × 2 (task performance: no mistakes vs. mistakes) ANOVA on the amount of autonomy yielded significant main effects of gaze, F(1,140) = 4.13, p = .044, ηp2= .03, indicating that when the robot followed the balls with its gaze was judged as more autonomous (M = 5.04, SD = 1.57) than a robot that looked straight at the conveyor (M = 4.48, SD = 1.85). The main effect of mistakes was also significant, F(1,140) = 9.61, p = .002, ηp2 = .06, which shows that when the robot did not make mistakes was judged as more autonomous (M = 5.16, SD = 1.52) than when the robot did make mistakes (M = 4.33, SD = 1.82). Next to these significant main effects, the hesitate×mistakes interaction effect was marginally significant, F(1,140) = 3.56, p = .061, ηp2 = .02. Post hoc t-tests showed the robot was judged as having more agency when it did not make mistakes (M=5.53, SD = 1.32) than when it did make mistakes (M=4.18, SD = 1.95) when it did not display hesitating behavior, t(77) = 3.61, p < .001, r = .14. This effect was absent when the robot does hesitate, t(75) =0.81 , p =.420. There difference in agency between hesitating (M=4.80, SD = 1.63) and no hesitating behavior (M=5.53, SD = 1.32) was significant when the robot did not make mistakest(78) = 2.18, p = 0.032 r = .06, while the effect was absent when the robot did make mistakes, t(74) = -0.72, p = .473. See also Figure 2.

No other effects were found (all ps > .100).

Figure 2:Bar plot of the mistakes x hestate interaction effect on agency. Error bars show the 95% confidence interval

Robot-like behavior

A univariate 2 (gaze behavior: no look vs. look) × 2 (motion fluency: smooth vs. shake) × 2 (hesitation: no hesitate vs. hesitate) × 2 (task performance: no mistakes vs. mistakes)ANOVA of the participants’ judgements on how robot-like they judged the behavior of the robot yielded significant main effects of all manipulated dimensions of behavior, see Table 4. In general, the introduction of gaze behavior, trembling motions, hesitations and mistakes made the robot less robot-like.No other effects were found (all ps> .100)

Effect / F(1,140) / p / ηp2 / M (SD) off / M (SD) on
gaze / 4.02 / .047 / .03 / 4.22 (1.94) / 3.69 (1.63)
shake / 8.07 / .005 / .05 / 4.33 (1.86) / 3.58 (1.69)
hesitate / 12.19 / .001 / .08 / 4.41 (1.73) / 3.48 (1.77)
mistakes / 6.58 / .011 / .04 / 4.29 (1.77) / 3.61 (1.80)

Table 4: Significant effects of the robot-like-ness of the robot’s behavior.

1