RESEARCH
Research interests
-
Language acquisition, development, and learning
-
Oral discourse and pragmatic abilities (e.g., narrative, argumentative)
-
Multimodal integration of gesture and speech prosody
-
Multimodal language learning and development, and embodied cognition
-
Educational interventions
-
Implementation science
-
Teacher training
The bootstrapping (causal) role of non-referential gestures
in children's cognitive and complex linguistic skills
While research in gesture studies and language development has predominantly focused on the role of referential gestures––those that visually represent the properties of a referent and closely relate to the semantic content of speech––the value of non-referential gestures has received less attention. My research aims to shed light on how non-referential gestures may support the development of children's cognitive and higher-level linguistic abilities, such as oral narrative and argumentative skills. Through experimental studies with children, my research seeks to investigate the mechanisms underlying the learning effects of these gestures on children's oral communication across various learning contexts and discourse types. Understanding the relative significance of different co-speech gestures is crucial to unraveling how gestures impact language development and learning, especially in the context of oral linguistic abilities. For instance, to address this question, I conduct gesture-based training studies.
​
Relevant publications:
​
Llanes-Coromina, J., Vilà-Giménez, I., Kushch, O., Borràs-Comes, J., & Prieto, P. (2018). Beat gestures help preschoolers recall and comprehend discourse information. Journal of Experimental Child Psychology, 172(8), 168–188. doi: 10.1016/j.jecp.2018.02.004
​
Vilà-Giménez, I., Igualada, A., & Prieto, P. (2019). Observing storytellers who use rhythmic beat gestures improves children’s narrative discourse performance. Developmental Psychology, 55(2), 250–262. doi: 10.1037/dev0000604
​
Vilà-Giménez, I., & Prieto, P. (2018). Encouraging children to produce rhythmic beat gestures leads to better narrative discourse performances. In K. Klessa, J. Bachan, A. Wagner, M. KarpiÅ„ski, & D. ÅšledziÅ„ski (Eds.), Proceedings of the 9th International Conference on Speech Prosody (pp. 704–708). PoznaÅ„, Poland. doi: 10.21437/SpeechProsody.2018-143
​
​Vilà-Giménez, I., & Prieto, P. (2020). Encouraging kids to beat: Children's beat gesture production boosts their narrative performance. Developmental Science, 23(6), 1–14. doi: 10.1111/desc.12967
The predictive value of co-speech gestures
in children's oral narrative abilities
Research concerning the multimodal communicative repertoire in infancy has indicated that referential gestures serve as both precursors and predictors of children’s onset of linguistic milestones. My research seeks to broaden the scope of investigation to further understand the mechanisms through which gestures––particularly non-referential ones––contribute to developmental trajectories. Using longitudinal data, I investigate how gestures, produced at various developmental stages and within diverse discourse genres, can predict oncoming changes in oral narrative proficiency.
​
Relevant publications:
​
Vilà-Giménez, I., Dowling, N., Demir-Lira, Ö. E., Prieto, P., & Goldin-Meadow, S. (2021). The predictive value of non-referential beat gestures: Early use in parent-child interactions predicts narrative abilities at 5 years of age. Child Development, 92(6), 2335–2355. doi: 10.1111/cdev.13583
​
Vilà-Giménez, I., Demir-Lira, Ö. E., & Prieto, P. (2020). The role of referential iconic and non-referential gestures in children's narrative production: Iconics signal oncoming changes in speech. Proceedings of the 7th Gesture and Speech in Interaction (GESPIN). KTH Speech, Music & Hearing and Språkbanken Tal. Stockholm, Sweden.
​
Gesture-speech multimodal development and communication
(1) Pragmatic and temporal integration between prosody, gesture and speech in children's narrative discourse
How do children develop their multimodal (gesture and speech) linguistic strategies? This research explores the gesture–speech temporal alignment patterns in children’s narrative speech and examines the relationship between information structure (IS) and gesture referentiality in children’s narrative speech from a longitudinal perspective and by assessing the potential differences between different gestures. This research has been conducted using the "Audiovisual corpus of Catalan children's narrative discourse development" (CatND)​, which consists of narrative productions carried out by 83 children at two time points in development (when children were 5-6 years old, and then two years later).
(Vilà-Giménez et al., 2021)
Accessible at OSF.
​​
Relevant publications: ​
​​
Florit-Pons, J., Vilà-Giménez, I., Rohrer, P. L., & Prieto, P. (2023). Multimodal development in children's narrative speech: Evidence for tight gesture-speech temporal alignment patterns as early as 5 years old. Journal of Speech, Language, and Hearing Research, 66(3), 888–900. doi: 10.1044/2022_JSLHR-22-00451
​
Florit-Pons, J., Vilà-Giménez, I., Rohrer, P. L., & Prieto, P. (2020). The development and temporal integration of co-speech gesture in narrative speech: A longitudinal study. Proceedings of the 7th Gesture and Speech in Interaction (GESPIN). KTH Speech, Music & Hearing and Språkbanken Tal. Stockholm, Sweden.
​​
Rohrer, P. L., Florit-Pons, J., Vilà-Giménez, I. & Prieto, P. (2022). Children use non-referential gestures in narrative speech to mark discourse elements which update common ground. Frontiers in Psychology, 12:661339. doi: 10.3389/fpsyg.2021.661339​​​​​​
​
​
(2) Gesture-speech multimodal communication across child-directed and adult-directed narrative discourse
​
How do young adults adapt their prosody and gestures when communicating to children? This research investigates how young adults (future early childhood teachers) modify their prosody and gestures when communicating with children, using the "TEACH-TALK: An audiovisual corpus of future teachers’ child- and adult-directed narratives" (TEACH-TALK) audiovisual corpus. By comparing multimodal cues such as gesture and prosody in child-directed speech (CDS) and adult-directed speech (ADS) in Catalan, this research explores how young adults modify their communicative behaviors to suit different audiences, shedding light on the distinct strategies employed in interactions with children vs. adults.​​​​​​​​​​​
(Vilà-Giménez et al., 2025.)
Accessible at OSF.
Multimodal corpus analysis
Annotation of audiovisual corpora: MultiModal MultiDimensional (M3D) labeling scheme
​
We have developed the MultiModal MultiDimensional (M3D) labeling system that proposes a multidimensional approach to gesture analysis, in that researchers may code for dimensions of the form of the gesture, independent of its relationship with speech prosody, or of its various pragmatic meanings. At the same time, this labeling scheme challenges a simplistic definition of non-referential (beat) gestures as biphasic rhythmic non-meaningful gestures.
​
​
(Rohrer et al., 2023)
​Accessible at OSF.​
​
Check the M3D training materials here!
​​​
​
Relevant publications: ​
​
Prieto, P., Cravotta, A., Kushch, O., Rohrer, P., L., & Vilà-Giménez, I. (2018). Deconstructing beat gestures: a labelling proposal. In K. Klessa, J. Bachan, A. Wagner, M. KarpiÅ„ski, & D. ÅšledziÅ„ski (Eds.), Proceedings of the 9th International Conference on Speech Prosody (pp. 201–205). PoznaÅ„, Poland. doi: 10.21437/SpeechProsody.2018-41
​
Rohrer, P. L., Tüntüncübasi, U., Florit-Pons, J., Vilà-Giménez, I., Esteve-Gibert, N., Ren-Mitchell, A., Shattuck-Hufnagel, S., & Prieto, P. (2025). Multidimensional labeling of gesture in communication: the M3D proposal. Corpus Pragmatics. doi: 10.1007/s41701-025-00197-2
The development of children's pragmatic and prosodic abilities
We created a test that aims to assess pragmatic abilities from a multimodal perspective (e.g., by taking into account prosodic and gestural patterns). The test assessment thus focuses not only on the verbal cues (speech, prosody) but importantly, it also considers multimodal aspects of communication (facial expression, gestures, posture).
​
Relevant publications:
Pronina, M., Hübscher, I., Vilà-Giménez, I., & Prieto, P. (2021). Bridging the Gap Between Prosody and Pragmatics: The Acquisition of Pragmatic Prosody in the Preschool Years and Its Relation With Theory of Mind. Frontiers in Psychology, 12:662124. doi: 10.3389/fpsyg.2021.662124
​
Pronina, M., Hübscher, I., Vilà-Giménez, I., & Prieto, P. (2019). A new tool to assess pragmatic prosody in children: evidence from 3- to 4-year-olds. In S. Calhoun, P. Escudero, M. Tabain, & P. Warren (Eds.), Proceedings of the 19th International Congress of Phonetic Sciences, Melbourne, Australia 2019. Canberra, Australia: Australasian Speech Science and Technology Association Inc. ​​​​​
Pronina, M., Hübscher, I., Vilà-Giménez, I., & Prieto, P. (2021, June 30). Retrieved from osf.io/pyc34