The Effects of Text Structure on learners’ reading: A multimodal Perspective from eye-tracking

Eye-tracking
  • English learners face some special circumstances such as an incomplete knowledge of English – limited knowledge of the English writing system, limited vocabulary, insufficient familiarity with grammatical structures, missing strategies for reading English, etc.

  • Given these circumstances and as shown by the eye-tracking results, learners devoted their mental resources to decoding the information in written language and consequently ignored the photographs. Only 1 out of 5 participants devoted 20 fixations on the visual illustrations. 

  • The illustrative photographs placed alongside written text clearly did not produce a superior multimodal effect for intermediate and advanced learners.

  • These results indicate that readers who are less able decoders are also less able to integrate

    accompanying visuals with written language. Commonsense view is that pictures do not only draw readers’ attention to the content of the reading materials but most importantly enable them to make intelligent guessing of the content of the text (Baharani & Ghafournia, 2015), while any such advantage may disappear if readers do not engage integrate the various modes to derive a multimodal meaning of the text. 

  • It was postulated in theories of multimodal reading that the presence of more modes in a text could be advantageous for learners.

  • Cope and Kalantzis (2009) explained such intratextual image-language semiosis as parallelism – the ability to get the same information through different message carriers – synaesthesias – the ability to mode switch for comprehension needs – and incommensurability – the ability of one mode to illuminate meaning in another because the two modes have fundamentally different potentials for meaning-making.

  • Since learners face special challenges such as limited knowledge of the English writing system, limited vocabulary, insufficient familiarity with grammatical structures of English, etc. (Birch,2007), it was a reasonable expectation that they would take advantage of the presence of modes that are less symbolic and more iconic to achieve a greater comprehension of the text.

  • The complementary text condition, characterized by the interdependence of its compositional modes, appears as a disjointed text to learners not accustomed to its genre.

  • As illustrated by the eye-tracking results, Learners deployed considerable mental resources figuring out the logical order of information, a mental task that takes cognitive resources away from other comprehension tasks.

Copyright © 2022 Djiguimde. All rights reserved.