Non-verbal Communication in Dialogue: Alignment between Eyebrow Raises and Pitch Accents in English

0
369

Non-verbal Communication in Dialogue: Alignment between Eyebrow Raises and Pitch Accents in English Maria L. Flecha-Garcia ([email protected]) Linguistics and English Language, 14 Buccleuch Place, University of Edinburgh Edinburgh EH8 9LN, UK Keywords: Nonverbal communication; prosody; dialogue. start of the events as reference points. Evidence of alignment was provided by a distribution peaking at zero (mean = -0.063sec, sd =0.458). To evaluate further whether the BR occurred significantly closer to one of its two surrounding accents, a pairwise t-test (two-tailed) was used to compare the mean distance between the preceding accent and the BR with that between the BR and the following accent. A significant difference (t = 2.381, df=271, p < .05) provided evidence that the BR did not start randomly between two accents and tended to occur closer to the following one. A one-way ANOVA did not show an effect of the participant’s identity on the mean distance between the BR and its following accent (F (2,270) = 0.552, p=.57). Introduction When we engage in conversation, we use both verbal and non-verbal communication, including facial expressions. The latter have been investigated mainly in relation to the expression of emotion, but research into their connection with speech is still scarce. This is surprising, considering its relevance for psycholinguistic theories of speech production and for the development of multimodal communication systems. Some studies have associated eyebrow raising with intonation, but there is no strong empirical support. A few exceptions include studies with synthetic animations in Dutch that have reported a preference for short eyebrow raises to be aligned with pitch accents (Krahmer and Swerts, 2004). The purpose of the current study was to investigate eyebrow raising in dialogue to test the hypothesis that brow raises are aligned with pitch accents in English. Since pitch accents have roles in discourse, if such alignment exists brow raises may contribute to these linguistic roles. Discussion and Conclusion Brow raises in the recorded dialogues occurred remarkably close to a pitch accent, which was usually its following accent. To the best of my knowledge, this is the first study that provides evidence of alignment between brow raises and pitch accents in English. This coordination may reveal an emphasizing function for brow raising, which is in agreement with earlier suggestions by Krahmer and Swerts (2004) in their perception studies using synthetic animated faces. The present study is part of a larger project that, using a rigorous method, has been successful at finding some possible linguistic functions for eyebrow raises (Flecha- Garcia, 2006a,b). Method and Results Materials Six recorded task-oriented dialogues were studied (average duration = 369sec) in which four female native speakers of British English collaborated in pairs. The start and end of pitch accented syllables and of brow raises from three of the participants were annotated by the author (a brow raise was any upward movement, from a baseline neutral position, of at least one eyebrow). There were a total of 271 brow raises and 1858 pitch accents. Figure 1 shows a sample frame from one of the recordings. Figure 1: Sample frame from a video-recorded dialogue. Statistical Analysis and Results Brow raises (BRs) may be aligned with a preceding or a following pitch accent (total synchrony is not expected). Thus, a frequency distribution was plotted for the distance in seconds between BRs and their nearest accent, using the Acknowledgments I would like to thank Dr Ellen G. Bard and Prof. D. Robert Ladd for supervising this project. Thanks also to EPSRC for their funding support. References Flecha-Garcia, M. L. (2006a). Eyebrow raising, discourse structure, and utterance function in face-to-face dialogue. Proceedings of the 28th Annual Conference of the Cognitive Science Society (pp. 1311-1316). Mahwah, NJ: Lawrence Erlbaum Associates. Flecha-Garcia, M. L. (2006b). Eyebrow raising in dialogue: Discourse structure, utterance function, and pitch accents. Doctoral dissertation, Linguistics and English Language, University of Edinburgh, Edinburgh. Krahmer, E. & Swerts, M. (2004). More about brows: A cross-linguistic study via analysis-by-synthesis. In Z. Rutkay & C. Pelachaud (Eds.). From brows to trust: Evaluating embodied conversational agents. Dordrecht, The Netherlands: Kluwer Academic Publishers.