1
Supplementary material to “Quantifying facial expression synchrony in face-to-face dyadic interactions: Temporal dynamics of simultaneously recorded facial EMG signals”Journal of Nonverbal Behavior
Marcel Riehle; Jürgen Kempkensteffen; and Tania M. Lincoln
University of Hamburg, Hamburg, Germany
Corresponding author: Marcel Riehle,
Significance threshold
We suggest using a significance threshold for calculated correlation coefficients to reduce the influence of smaller non-significant correlations that emerge as a result of measurement error or by chance. We set our threshold by using Bonferroni correction to adjust for multiple testing for the number of correlation coefficients calculated within a WCLC matrix. In an example WCLC analysisfor a time series of 1000 observations, sampled at 50Hz, using a window length of 5s (= 250 observations) and a maximum time lag of 2s (= 100 observations), the resulting WCLC matrix would consist of 201 columns and 749 rows (cf. Boker et al., 2002). Thus, 150,549 correlation coefficients would be calculated. This leads to a significance threshold of padj≈ .0000003, when originally testing at an α-level of p < .05. Therefore, correlation coefficients would have to exceed a threshold of rcrit≈ .34. Note, that the degrees of freedom used for setting the threshold are bound to the numbers of data points within a window (in this example = 250 observations). For this reason, using the significance threshold is only advised when at least roughly 100 observations can be used as the window size, since the significance threshold is likely to asymptotically reach a value of 1 otherwise (cf. Figure S1).
Fig. S1Distribution of critical Pearson correlation values of the significance threshold alongside varying window sizes for a WCLC analysis on a 3 min data sequence, sampled at 32 Hz (matching this study; in total 5760 data points). The different lines represent different maximum time lags (this study used a window size of 224 and a maximum lag of 64). Note: max. lag = maximum time lag; numbers for window size and max. lag refer to numbers of observations
No negative correlation coefficients
We suggest that negative correlation coefficients should not be included in the WCLC resultsmatrix and considered equal to 0. The approach described by Ramseyer and Tschacher (2011) included negative correlation coefficients by means of including the absolute correlation coefficient values in the calculation of synchrony parameters. However, including negative correlations (and by this asynchrony between the time series) in the synchrony parameters requires a couple of assumptions to be met. First, one needs to assume that asynchrony is another form of synchrony. Second, one needs to assume that including asynchrony has a benefit over only including actual synchrony in that actual synchrony would not be detected by the analysis adequately otherwise.Because of the practical observation that high positive WCLC coefficients are commonly accompanied by high negative WCLC coefficients somewhere along the coefficient distribution of the same time window, we consider negative coefficients likely to emerge as an artifact of the time lagged analysis.Consider the example illustrated in Figure S2 or the example given in the main article, where either the high positive or the high negative correlation best represents the relationship between persons A and B, but not both do.Omitting negative values therefore increases the signal-to-noise ratio and the power to discriminate between leading and pacing roles in the interaction.
1
Fig. S2Cross-lagged correlation (CLC) distribution (middle panel) and the corresponding range corrected EMG time series window (top & bottom panel) of person A (black line) and person B (grey lines). The dashed line in the top panel illustrates person B’s time series shifted according to the time lag corresponding to the maximum CLC (maximum synchrony) value (at +0.25s). The dashed line in the bottom panel illustrates person B’s time series shifted according to the time lag corresponding to the minimum CLC (maximum asynchrony) value (at+2.00s). Almost perfect synchrony (CLC = .99) as well as asynchrony (CLC = -.95) can be observed within a single time window
1
Parameter change sensitivity
As shown in our manuscript, our WCLC approach at sample level does a good job in detecting smiling synchrony when comparing genuine and pseudo-interactions. To illustrate what high or low parameter producing time series actually look like on the dyad level, we have illustrated three series included in our analysis in Figure S3. We have included the time series of a particularly low synchrony dyad (dyad #10; mean overall synchrony score = 0.13) and a particularly high synchrony dyad (dyad #11; mean overall synchrony score = 0.52). For comparison, we have also included the respective pseudo-interaction of dyad #11 (mean overall synchrony score = 0.10). Visual inspection of the convergence of the time series clearly confirms that the two time series of dyad #11 are more synchronous than the time series of dyad #10 and also the time series of the pseudo-interaction of dyad #11. Accordingly, the WCLC parameter distribution for dyad #11 contains much higher values (particularly at time lags closer to 0) than the other two interactions. This shows two important properties of the WCLC analysis: First, the analysis is able to distinguish dyads with large amounts of synchrony from dyads with low amounts of synchrony and thus generates variance. Second, high values of synchrony are not merely generated by one (or both) interaction partners being highly expressive, but by the interaction partners actually synchronizing their movements. The latter is supported by the within dyad difference between the genuine and pseudo-interaction of dyad #11, as it contains one of the two interaction partners (i.e. the recipient), who is considerably high in expressiveness.
1
Fig. S3 Illustration of three of the analyzed time series pairs (smiling/zygomaticus; left) and their respective WCLC parameter distribution (right)
1
Additional conversational content variables
Inspection of the emotion ratings and event titles for the positive and negative events revealed that for the positive events all answers were either “happiness” (88%) or “surprise” (10%) (1 missing). Participants who had given a “surprise” answer had also used a rating of 8 or 9 on the happiness intensity rating indicating that it was indeed happiness that was felt. For the negative emotions, there was some more variety with primary emotions varying between anger (73%), sadness (13%), fear (5%), neutral/other (3%), and surprise (2%) (2 missings). Events that were not rated with primary anger were mostly related either to job loss or job uncertainty or to interpersonal arguments. With the exception of one event, which related to the loss of a grandmother and might have not actually tapped anger, all events were potentially “annoying” (as also indicated by high anger intensity ratings).“
Comparing WCLC with and without constraints
In an intention to compare our approach and the approach used by Ramseyer and Tschacher (2011) more directly, we calculated the WCLC synchrony without significance threshold and without excluding absolute values of negative correlations.We thenran the same three-way repeated measures ANOVA as described in our manuscript (muscle site by interaction type by (interactional) phase) on the unconstrained WCLC values.
The results showed significant main effects of muscle site (F(1,29) = 110.0; p < .001; ηP²= .79), interaction type (F(1,29) = 13.8; p = .001; ηP²= .32), and phase (F(1,29) = 8.5; p < .001; ηP²= .23). Significant interactions were found for muscle site by interaction type (F(1,29) = 6.6; p = .02; ηP²= .19) and muscle site by phase (F(1, 29) = 15.1; p < .001; ηP² = .34). Smiling (cf. Figure S4) but not frowning synchrony (cf. Figure S5) in genuine interactions could be distinguished from chance level synchrony (pseudo-interactions). Paired samples t-tests confirmed this differential effect for all of the six interactional phases (smiling: ts(29) = 2.32–3.35; ps = .002–.03; dzs = 0.42–0.61; frowning: ts(29) = 0.06–1.76; ps = .09–.95; dzs = 0.01–0.32).
Thus, the overall effects were comparable using an approach with and without significance threshold plus omitting negative correlations. However, the mean noise-levels were higher in the unconstrained WCLC analysis and signal-to-noise ratio was considerably lower. This favors the usage of a significance threshold as well as omitting negative correlations from the WCLC matrix.
Fig. S4Distribution of mean (zygomaticus) smiling synchrony unconstrained WCLC estimates for the along the different time-lags for each interactional phase for both genuine (solid line) and pseudo-interactions (dotted line). The shaded gray areas represent ±1 standard error of the mean
1
Fig. S5 Distribution of mean (corrugator) frowning synchrony unconstrained WCLC estimates for the along the different time-lags for each interactional phase for both genuine (solid line) and pseudo-interactions (dotted line). The shaded gray areas represent ±1 standard error of the mean
1
References
Boker, S. M., Rotondo, J. L., Xu, M., & King, K. (2002). Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series. Psychological Methods, 7(3), 338–355. doi:10.1037/1082-989X.7.3.338
Ramseyer, F., & Tschacher, W. (2011). Nonverbal synchrony in psychotherapy: coordinated body movement reflects relationship quality and outcome. Journal of Consulting and Clinical Psychology, 79(3), 284–95. doi:10.1037/a0023419