Article

Name: Emotion recognition by speech signal characteristics (linguistic, clinical, informative aspects)

Authors: L. P. Prokofyeva, I. L. Plastun, N. V. Filippova, L. Yu. Matveeva, Na. S. Plastun

Saratov State Medical University, Saratov, Russian Federation; Saratov State Technical University, Saratov, Russian Federation; Higher School of Economics, Moscow, Russian Federation

In the section Linguistics

Abstract: The paper presents an experimental project of linguists, medical professionals, lawyers, computer security specialists dealing with emotions discernment by basic speech signal character-istics. The software has been created, and its first testing has been carried out in the social network VKontakte. The collected recordings of speech fragments of live spontaneous prox-imate-intermediated dialogical speech were analyzed at several levels. First, a complex linguistic analysis revealed lexico-semantic and prosodic features of emotionality. Then, a comparison with the software results was carried out, and the data obtained were systematized. Also, conclusions on the leading role of prosody in revealing hidden types of emotional stress were made. Frequent agreements of digital meanings of prosodic elements in speech segments were found demonstrating emotions not fixed at the lexico-semantic level. Finally, the working formulation for external and internal ways of emotionality expression in the live speech was offered.

Keywords: emotions, prosodics, speech recognition, experiment, computer program

Bibliography:

Damasio A. R. Descartes’ error: Emotions, reason, and the human brain. New York, NY, Putnam’s Sons, 1994.

Filippova N. V., Baryl’nik Yu. B., Antonova A. A., Deeva M. A., Guseva M. A. Rol' psikholingvisticheskikh metodov v diagnostike narusheniy rechevogo i umstvennogo razvitiya u detey [The role of psycholinguistic methods in the diagnosis of disorers of speech and mental development in children]. Russian Journal of Psychiatry. 2016, no. 5, pp. 555–561.

Ganzin I. V. Klinicheskaya lingvistika: kompleksnaya diagnostika rechevogo povedeniya pri psikhicheskikh rasstroystvakh [Clinical linguistics: complex diagnosis of speech behavior in mental disorders]. Simferopol’, Feniks, 2011, 400 p.

Ganzin I. V. Osnovy psikhosomatiki. Psikhobiosotsial'naya model': Integrativnoe ucheb. posobie [Fundamentals of psychosomatics. Psychobiosocial model: an integrative textbook]. Simferopol’, Dolya, 2012, 184 p.

Hopkins C. S., Ratley R. J., Benincasa D. S., Grieco J. J. Evaluation of voice stress analysis technology. In: Proceedings of the 38th Annual Hawaii International Confer-ence on System Sciences. Hawai: IEEE, 2005. URL: https://ieeexplore.ieee.org/ document/1385268 (accessed: 22.05.2019).

Kiselev V. V. Avtomaticheskoe opredelenie emotsiy po rechi [Automatic determination of emotions by speech]. Obrazovatelnye Tekhnologii. 2012, no. 3, pp. 85–89.

Lövheim H. A new three-dimensional model for emotions and monoamine neuro-transmitters. Med. Hypotheses. 2012, no. 78, pp. 341–348.

Matveeva L. Yu., Prokof’eva L. P. Zvukovoe otrazhenie emotsiy (instrumental’nye metody v fonoskopicheskoy ekspertize) [Sound reflection of emotions (instrumental methods in phonoscopic expertise)]. Izvestiya of Saratov University. New Series. Series: Philology. Journalism. 2016, vol. 16, no. 2, pp. 152–155.

Plastun I. L., Zakharov A. A., Eremin N. A. Avtomatizirovannaya sistema dlya opredeleniya psikhologicheskogo sostoyaniya cheloveka po dinamike rechevogo signala [Automated system for determining the psychological state of a person by the dynamics of the speech signal]. Sotsial’no-gumanitarnye tekhnologii. 2018, no. 1 (5), pp. 74–78.

Sokolov E. N. Sfericheskaya model’ intellektual’nykh operatsiy[Spherical model of intellectual operations]. Psikhologicheskii Zhurnal. 2001, vol. 22, no. 3, pp. 49–56.

Vartanov A. V. Antropomorfnyy metod raspoznavaniya emotsiy v zvuchashchey rechi [The anthropomorphic method of emotion recognition in audio speech]. National Psychological Journal. 2013, no. 2 (10), pp. 69–79.

Velichkova L. V. Psikholingvisticheskaya osnova issledovaniya emotsional'nosti zvuchashchey rechi [Psycholinguistic basis of research of emotionality of sounding speech]. Journal of Psycholinguistics. 2007, no. 5, pp. 21–27.

Yang N. Algorithms for Affective and Ubiquitous Sensing Systems and for Protein Structure Prediction. New York, 2015, 186 p.