Linking production and comprehension – Investigating the lexical interface

Abstract

In a typical conversation, listening and speaking go hand in hand. However, we do not yet know what cognitive machinery and brain areas are shared between production and comprehension. It is believed that the lexical (lemma) and conceptual levels are shared between listening and speaking (Levelt et al., 1999). In this project, we test these claims using behavioural and neuroimaging methods.
Cumulative semantic interference effects have been shown in picture naming using a continuous naming paradigm (Howard et al., 2006). A linear increase in picture naming latencies has been found for subsequent exemplars of a semantic category (e.g., fork, spoon, knife, etc.) when pictures from different categories are mixed together (e.g., fork, horse, desk, spoon, rocket, pig, chair, knife, etc.). Some models claim that the effect originates in conceptual to lexical mappings (Howard et al., 2006; Oppenheim et al., 2010) while WEAVER++ claims the effect lies at the conceptual level (Roelofs, 2018). As these levels are claimed to be shared between production and comprehension, mechanisms similar to those underlying cumulative semantic effects in picture naming should also be present during speech comprehension and thus, we should observe these effects in a listening task involving conceptual and lexical levels.
We used four tasks – bare picture naming and gender-marked picture naming (production), and semantic classification and gender classification on spoken words (comprehension) - in a within-subject (N=32) experiment, conducted in Dutch, using a continuous naming paradigm (Howard et al., 2006). Based on the WEAVER++ model, we expected to replicate the cumulative semantic interference effect found in the bare picture naming task in previous studies and predicted cumulative semantic facilitation in the semantic classification task. We included the gender-marked picture naming and gender classification tasks because gender, a syntactic property, is believed to be associated with the lemma in gender-marked languages. We expected a cumulative semantic interference effect on the gender-marked picture naming task and no effect on the gender classification task.
We replicated the cumulative semantic interference effect on the bare picture naming task. A cumulative semantic facilitation effect was found in the semantic classification task. We found no cumulative semantic effects in either the gender classification task or the gender-marked picture-naming task. The findings on semantic classification support WEAVER++ model but challenge other models of cumulative semantic effects (Howard et al., 2006; Oppenheim et al., 2010) because they currently do not include mechanisms to account for such effects in comprehension. From a neural perspective, there is evidence suggesting that lemmas are likely localised in the middle portion of the left middle temporal gyrus (left mMTG) (Dronkers et al., 2004; Indefrey and Levelt, 2004; Indefrey, 2011; Piai et al., 2014; Schwartz et al., 2009). To establish this empirically, we conducted an fMRI study with the four tasks that were used in the behavioural study. We are acquiring data from the last two (of 32 participants). Results of the fMRI data will be discussed. We predict that the conjunction of activation across the four tasks will be localized in the left mMTG.