Article

Name: Multimodal Communication in the Focus of Research Reflection: A Problem-Oriented Approach

Authors: Anna V. Beloedova, Evgeny A. Kozhemyakin

Belgorod State National Research University, Belgorod, Russian Federation

Abstract: The paper discusses the problem-oriented approach to the multimodal communication as it is expounded by John Bateman, Janina Wildfeuer and Tuomo Hiippala. The authors claim for the productivity of the approach to the analysis of the contemporary informational communicational system in terms of developing media, i.e. smart- phones, computer games, 3D cinema, VR and AR technologies, etc., which are multimodal by their nature and use. The basic notions associated with multimodality include mode, communication situation, ergodic text with its features of mutability and temporality. The main characteristics of communication situations as an item of the multimodal analysis is the source of information, recipients, conditions of communication and its duration. While interacting, the communication situations bear linear, micro-ergodic, non-mutable ergodic and mutable ergodic types depending on the prevailing way of perception. The authors claim that the most representative results of research of “vague” and unpredictable multimodal objects could be gained by the mixt of quantitative and qualitative methods, including corpus analysis, eye-tracking technics, and computing methods.

Keywords: multimodality, multimodal studies, problem-oriented approach, basic principles of multimodality, communication situation, multimodal research methods

Bibliography:

Bateman J., Wildfeuer J., Hiippala T. Multimodality. Berlin, Boston, De Gruyter Mouton, 2017.

Bateman J., Wildfeuer J., Hiippala T. Book Review: A question of definitions: foundations for multimodality: A response to Charles Forceville’s review. Visual Communication, 2020, no. 19 (2), pp. 317–320.

Degteva I. S. J. Bateman, J. Wildfeuer, T. Hiippala. Mul'timodal'nost'. Bazovye printsipy, sposoby issledovaniya i metody analiza. Problemno-orientirovannoe vvedenie. (Referat). In: Ilin M. V. et al. (eds.). METOD: Moskovskiy ezhegodnik trudov iz obshchestvovedcheskikh distsiplin. Moscow, 2019, iss. 9, pp. 92–97. (in Russ.)

Featherman C. Book review: John Bateman, Janina Wildfeuer and Tuomo Hiippala. Multimodality: Foundations, Research and Analysis: A Problem-Oriented Introduction. Discourse Studies, 2018, no. 20 (3), pp. 443–445.

Forceville C. Book Review: Multimodality: Foundations, Research and Analysis – A Problem-Oriented Introduction. Visual Communication, 2020, no. 19 (1), pp. 157–160.

Hockings P. (ed.). Principles of visual anthropology. Berlin, Walter de Gruyter, 2003.

Kibrik A. A. Mul'timodal'naya lingvistika. Kognitivnye issledovaniya, 2010, vol. 4, pp. 134–152. (in Russ.)

Kress G. R., Leeuwen T. van. Multimodal discourse: the modes and media of contemporary communication. London, Arnold, 2001.

Matsumoto D., Frank M. G., Hwang H. S. (eds.). Nonverbal communication: Science and applications. London, Sage Publ., 2012.

Moriarty S. Visual semiotics theory. Handbook of visual communication: Theory, methods, and media, 2005, no. 8, pp. 227–241.

Nöth W. Visual semiotics: Key features and an application to picture ads. The Sage handbook of visual research methods, 2011, pp. 298–316.

Poyatos F. (ed.). Advances in nonverbal communication: sociocultural, clinical, esthetic and literary perspectives. London, John Benjamins Publ., 1992.

Zagar I. Against Visual Argumentation: Multimodality as Composite Meaning and Composite Utterances. In: Mohammed D., Lewinski M. (eds.). Argumentation and Reasoned Action. College Publications, 2016, pp. 829–852.