Attention/physiology; Eye Movement Measurements; Fixation, Ocular/physiology; Humans; Language; Psychomotor Performance; Reaction Time; Saccades/physiology; Space Perception/physiology; Speech Perception/physiology; Visual Perception/physiology
Abstract :
[en] Language is more than a source of information for accessing higher-order conceptual knowledge. Indeed, language may determine how people perceive and interpret visual stimuli. Visual processing in linguistic contexts, for instance, mirrors language processing and happens incrementally, rather than through variously-oriented fixations over a particular scene. The consequences of this atypical visual processing are yet to be determined. Here, we investigated the integration of visual and linguistic input during a reasoning task. Participants listened to sentences containing conjunctions or disjunctions (Nancy examined an ant and/or a cloud) and looked at visual scenes containing two pictures that either matched or mismatched the nouns. Degree of match between nouns and pictures (referential anchoring) and between their expected and actual spatial positions (spatial anchoring) affected fixations as well as judgments. We conclude that language induces incremental processing of visual scenes, which in turn becomes susceptible to reasoning errors during the language-meaning verification process.
Disciplines :
Theoretical & cognitive psychology
Author, co-author :
Dumitru, Magdalena ; Université de Liège > Département de Psychologie > Psychopathologie cognitive
Joergensen, Gitte H.
Cruickshank, Alice G.
Altmann, Gerry T. M.
Language :
English
Title :
Language-guided visual processing affects reasoning: the role of referential and spatial anchoring.
Publication date :
2013
Journal title :
Consciousness and Cognition
ISSN :
1053-8100
eISSN :
1090-2376
Publisher :
Elsevier, Atlanta, United States - California
Volume :
22
Issue :
2
Pages :
562-71
Peer reviewed :
Peer Reviewed verified by ORBi
Commentary :
Copyright (c) 2013 Elsevier Inc. All rights reserved.
Allopenna P.D., Magnuson J.S., Tanenhaus M.K. Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language 1998, 38(4):419-439.
Altmann G.M.T. Language-mediated eye movements in the absence of a visual world: The blank screen paradigm. Cognition 2004, 93:79-87.
Altmann G.T.M., Kamide Y. Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition 1999, 73:247-264.
Barsalou L.W. Perceptual symbol systems. Behavioral and Brain Sciences 1999, 22:577-660.
Braine M., Rumain B. Development of comprehension of 'Or': Evidence for a sequence of competencies. Journal of Experimental Child Psychology 1981, 31(1):46-70.
Brysbaert M., New B. Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved frequency measure for American English. Behavior Research Methods 2009, 41(4):977-990.
Chevallier C., Noveck I.A., Nazir T., Bott L., Lanzetti V., Sperber D. Making disjunctions exclusive. The Quarterly Journal of Experimental Psychology 2008, 61(11):1741-1760.
Chevallier C., Wilson D., Happé F., Noveck I. Scalar inferences in autism spectrum disorders. Journal of Autism and Developmental Disorders 2010, 10.1007/s10803-010-0960-8.
Cooper R.M. The control of eye fixation by the meaning of spoken language. A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology 1974, 6(1):84-107.
Demarais A., Cohen B.H. Evidence for image-scanning eye movements during transitive inference. Biological Psychology 1998, 49:229-247.
Dils A.T., Boroditsky L. Processing unrelated language can change what you see. Psychonomic Bulletin and Review 2010, 17(6):882-888.
Egly R., Driver J., Rafal R.D. Shifting visual attention between objects and locations: Evidence from normal and parietal lesion subjects. Journal of Experimental Psychology: General 1994, 123:161-176.
Fillenbaum S. Or: Some uses. Journal of Experimental Psychology 1974, 103(5):913-921.
Glenberg, A. M., & Kaschak, M. P. (2003). The body's contribution to language. In B. H. Ross (Ed.), The Psychology of Learning and Motivation. Advances in Research and Theory (Vol. 43, pp 93-126). New York: Academic Press.
Johansson R., Holsanova J., Holmqvist K. Pictures and spoken descriptions elicit similar eye movements during mental imagery, both in light and in complete darkness. Cognitive Science 2006, 30(6):1053-1080.
Kamide Y., Altmann G.M.T., Haywood S.L. The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language 2003, 49:133-159.
Kashak M.P., Glenberg A.M. Constructing meaning: The role of affordances and grammatical constructions in sentence comprehension. Journal of Memory & Language 2000, 43:508-529.
Kaschak M.P., Madden C.J., Therriault D.J., Yaxley R.H., Aveyard M., Blanchard A.A., Zwaan R.A. Perception of motion affects language processing. Cognition 2005, 94:B79-B89.
Lamy D., Egeth H. Object-based selection: The role of attentional shifts. Perception and Psychophysics 2002, 64:52-66.
Link G. The logical analysis of plurals and mass terms: A lattice-theoretical approach. Meaning, use, and interpretation of language 1983, 302-323. de Gruyter, Berlin. R. Baeuerle (Ed.).
Matlock T. Fictive motion as cognitive simulation. Memory and Cognition 2004, 32:1389-1400.
Paris S.G. Comprehension of language connectives and propositional logical relationships. Journal of Experimental Child Psychology 1973, 16(2):278-291.
Richardson D., Matlock T. The integration of figurative language and static depictions: An eye movement study of fictive motion. Cognition 2007, 102:129-138.
Richardson D.C., Spivey M.J., Barsalou L.W., McRae K. Spatial representations activated during real-time comprehension of verbs. Cognitive Science 2003, 27:767-780.
Rossion B., Pourtois G. Revisiting Snodgrass and Vanderwart's object pictorial set: The role of surface detail in basic-level object recognition. Perception 2004, 33(2):217-236.
Simons M. Disjunction and alternativeness. Linguistics and Philosophy 2001, 24:597-619.
Snodgrass J.G., Vanderwart M. A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory 1980, 6(2):174-215.
Spivey M.J., Geng J.J. Oculomotor mechanisms activated by imagery and memory: Eye movements to absent object. Psychological Research 2001, 65:235-241.
Stanfield R.A., Zwaan R.A. The effect of implied orientation derived from verbal context on picture recognition. Psychological Science 2001, 12(2):153-156.
Tanenhaus M.K., Spivey-Knowlton M.J., Eberhard K.M., Sedivy J.C. Integration of visual and linguistic information in spoken language comprehension. Science 1995, 268:1632-1634.
Tversky A., Kahneman D. Judgment under uncertainly: Heuristics and biases. Science 1974, 185:1124-1130.
Zwaan, R. A. (2004). The immersed experience. Toward an embodied theory of language comprehension. In B. H. Ross (Ed.), The psychology of learning and motivation (vol. 44, pp. 35-62). New York: Academic Press.
