RESEARCH PROJECTS

My PhD dissertation (directed by Dr. Pilar Prieto, ICREA/UPF) examined several issues related to the bootstrapping role of non-referential beat gestures —and their concomitant prosodic prominence— in 5- to 6-year-old children’s narrative abilities, focusing on the multimodal integration of gesture, speech and prosody language development.

BEAT GESTURES AND

CHILDREN'S NARRATIVE PERFORMANCE

  • The potential effect of encouraging the production of beat gestures on children's narrative discourse performance (i.e., narrative skills) in a brief narrative discourse training.

Vilà-Giménez, I., & Prieto, P. (2020). Encouraging kids to beat: Children's beat gesture production boosts their narrative performanceDevelopmental Science, 23(6), 1–14. doi: 10.1111/desc.12967

Vilà-Giménez, I., & Prieto, P. (2018). Encouraging children to produce rhythmic beat gestures leads to better narrative discourse performances. In K. Klessa, J. Bachan, A. Wagner, M. Karpiński, & D. Śledziński (Eds.), Proceedings of the 9th International Conference on Speech Prosody (pp. 704–708). Poznań, Poland. doi: 10.21437/SpeechProsody.2018-143

E-News (UPF)

La Vanguardia

Art in Real Life - Blog

EurekAlert!

MedicalXpress

  • The effects of beat gestures' observation on children's narrative discourse comprehension.

Llanes-Coromina, J., Vilà-Giménez, I., Kushch, O., Borràs-Comes, J., & Prieto, P. (2018). Beat gestures help preschoolers recall and comprehend discourse informationJournal of Experimental Child Psychology, 172(8), 168–188. doi: 10.1016/j.jecp.2018.02.004

E-News (UPF)

La Vanguardia

Diari de Girona

  • The potential effect of observing beat gestures on children's narrative discourse performance (i.e., narrative skills) in a brief narrative discourse training.

Vilà-Giménez, I., Igualada, A., & Prieto, P. (2019). Observing storytellers who use rhythmic beat gestures improves children’s narrative discourse performanceDevelopmental Psychology, 55(2), 250–262. doi: 10.1037/dev0000604

E-News (UPF)

La Vanguardia

The Global Source for Science News

BEAT GESTURES: LABELING PROPOSAL

Together with my colleagues Dr. Alice Cravotta, Patrick L. Rohrer, Dr. Olga Kushch and Dr. Pilar Prieto, we advanced a comprehensive gesture labelling proposal which highlights the independence of the prosodic and semantic properties of different gesture types and at the same time challenges a simplistic definition of beat gestures as biphasic rhythmic non-meaningful gestures.

 

Prieto, P., Cravotta, A., Kushch, O., Rohrer, P., & Vilà-Giménez, I. (2018). Deconstructing beat gestures: a labelling proposal. In K. Klessa, J. Bachan, A. Wagner, M. Karpiński, & D. Śledziński (Eds.), Proceedings of the 9th International Conference on Speech Prosody (pp. 201–205). Poznań, Poland. doi: 10.21437/SpeechProsody.2018-41

 

Together with Patrick L. Rohrer, Júlia Florit-Pons, Glenda Gurrado, Dr. Núria Esteve-Gibert, Ada Ren, Dr. Stefanie Shattuck-Hufnagel and Dr. Pilar Prieto we developed a labeling system that proposes a multidimensional approach to gesture analysis, in that researchers may code for dimensions of the form of the gesture, independent of its relationship with speech prosody, or of its various pragmatic meanings.

Rohrer, P. L., Vilà-Giménez, I., Florit-Pons, J., Esteve-Gibert, N., Ren, A., Shattuck-Hufnagel, S., & Prieto, P.  (2020). The MultiModal MultiDimensional (M3D) labelling scheme for the annotation of audiovisual corpora. Proceedings of the 7th Gesture and Speech in Interaction (GESPIN). KTH Speech, Music & Hearing and Språkbanken Tal. Stockholm, Sweden.

In collaboration with:

  • Patrick L. Rohrer (Universitat Pompeu Fabra)

  • Júlia Florit-Pons (Universitat Pompeu Fabra)

  • Dr. Núria Esteve-Gibert (Universitat Oberta de Catalunya)

  • Ada Ren (Speech Communication Group at the Research Laboratory of Electronics, Massachusetts Institute of Technology, USA)

  • Dr. Stefanie Shattuck-Hufnagel (Speech Communication Group at the Research Laboratory of Electronics, Massachusetts Institute of Technology, USA)

  • Dr. Pilar Prieto (ICREA; Universitat Pompeu Fabra)

 

The MultiModal MultiDimensional (M3D) labeling manual represents a joint effort between three gesture labs (Language and Cognition Research Group, UOC; Speech Communication Group, MIT; Prosodic Studies Group, UPF) to design a comprehensive guide for the annotation of multimodal speech corpora, particularly focusing on co-speech gestures, their relationship with prosody, and their semantic and pragmatic properties.

Related projects:

 

Florit-Pons, J., Vilà-Giménez, I., Rohrer, P. L., & Prieto, P. (in prep.). A longitudinal examination into the development and temporal integration of co-speech gesture in children's narrative speech.
Florit-Pons, J., Vilà-Giménez, I., Rohrer, P. L., & Prieto, P.  (2020). The development and temporal integration of co-speech gesture in narrative speech: A longitudinal study. Proceedings of the 7th Gesture and Speech in Interaction (GESPIN). KTH Speech, Music & Hearing and Språkbanken Tal. Stockholm, Sweden.

In collaboration with:

  • Júlia Florit-Pons (Universitat Pompeu Fabra)

  • Patrick L. Rohrer (Universitat Pompeu Fabra)

  • Dr. Pilar Prieto (ICREA; Universitat Pompeu Fabra)

Together with my colleagues Júlia Florit-Pons, Patrick L. Rohrer and Dr. Pilar Prieto, we carried out a longitudinal study that aims to assess gesture development and gesture-speech alignment in children's narrative discourse. This study uses a longitudinal database of 332 narratives performed by 83 Catalan children at two periods in development (at 5-6 and two years later) (Vilà-Giménez, I., Florit-Pons, J., Gurrado, G., Rohrer, P. L., & Prieto, P.'s audiovisual corpus of Catalan children's narrative speech development)Results revealed that the use of non-referential gestures (as opposed to referentials) increases significantly in development and that by age 5-6, children already temporally integrate gesture with speech prominence.

CHILDREN'S PRAGMATIC

AND PROSODIC ABILITIES

Pronina, M., Hübscher, I., Vilà-Giménez, I., & Prieto, P. (in preparation). The acquisition of prosody in the preschool years: Bridging the gap between prosody and pragmatics.
Pronina, M., Hübscher, I., Vilà-Giménez, I., & Prieto, P. (2019). A new tool to assess pragmatic prosody in children: evidence from 3- to 4-year-olds. In S. Calhoun, P. Escudero, M. Tabain, & P. Warren (Eds.), Proceedings of the 19th International Congress of Phonetic Sciences, Melbourne, Australia 2019. Canberra, Australia: Australasian Speech Science and Technology Association Inc. 

In collaboration with:

  • Mariia Pronina (Universitat Pompeu Fabra)

  • Dr. Iris Hübscher (University of Zurich)

  • Dr. Pilar Prieto (ICREA; Universitat Pompeu Fabra)

 

We created a test that aims to assess pragmatic abilities from a multimodal perspective (e.g., by taking into account prosodic and gestural patterns). The test assessment thus focuses not only on the verbal cues (speech, prosody), but importantly, it also considers multimodal aspects of communication (facial expression, gestures, posture). 

THE PREDICTIVE ROLE OF BEAT GESTURES ON CHILDREN'S NARRATIVE DEVELOPMENT

Vilà-Giménez, I., Dowling, N., Demir, Ö. E., Prieto, P., & Goldin-Meadow, S. (submitted). The predictive value of beat gestures: Early use predicts narrative abilities at 5 years of age.

In collaboration with:

  • Natalie Dowling (University of Chicago, IL, USA)

  • Dr. Ö. Ece Demir-Lira (University of Iowa, IL, USA)

  • Dr. Pilar Prieto (ICREA; Universitat Pompeu Fabra)

  • Prof. Susan Goldin-Meadow (University of Chicago, IL, USA)

We conducted a longitudinal study which analyzes the speech and gestures produced by 45 14- to 58-months-old children in naturalistic interactions with their caregivers. Results showed that the early spontaneous production of non-referential beat gestures, as opposed to the production of referential iconic gestures and non-referential hand flip gestures, predicts later narrative abilities at 5 years of age. We also conducted a very detailed analysis of children's target sentences with co-speech gestures (beat, flip and iconic gestures) following a pragmatic coding scheme.

Vilà-Giménez, I., Demir-Lira, Ö. E. , & Prieto, P.  (2020). The role of referential iconic and non-referential beat gestures in children's narrative production: Iconics signal oncoming changes in speech. Proceedings of the 7th Gesture and Speech in Interaction (GESPIN). KTH Speech, Music & Hearing and Språkbanken Tal. Stockholm, Sweden.

In collaboration with:

  • Dr. Ö. Ece Demir-Lira (University of Iowa, IL, USA)

  • Dr. Pilar Prieto (ICREA; Universitat Pompeu Fabra)

Following a longitudinal approach, this study uses an audiovisual corpus of natural narrative productions at two point times in development (5-6 and two years later) (Vilà-Giménez, I., Florit-Pons, J., Gurrado, G., Rohrer, P. L., & Prieto, P.'s audiovisual corpus of Catalan children's narrative speech development).

The study aims to assess the predictive value of gestures in children's narrative development by comparing the effects of referential iconic gestures and non-referential beat gestures produced in narrative corpora. Specifically, we investigate whether iconic gestures and/or beat gestures performed by 5- to 6-year-old children while producing narratives can signal changes in children's later narrative performance two years later. Results demonstrated that referential iconics (as opposed to non-referential beat gestures) that accompanied narratives produced by children at 5 and 6 years of age are significant predictors of later narrative scores two years later.

Follow on social

© 2020 by Ingrid Vilà-Giménez.
 

  • LinkedIn - Círculo Negro
  • Twitter - Círculo Negro