Emotional expression in the digital realm


Emotional survival during pandemic.

Fortunately, the COVID pandemic is gradually declining around the world, cases are diminishing, we can go out safely for shopping, travel, spare time at restaurants, bars, and theaters. Now we can leave out our confinement at home and leave behind the “home office”.  It seemed like an “eternity”, on that time during the first months of 2020, when all around the world suddenly our social and emotional space were reduced, confined to closed spaces, forcing us to a minimum face to face contact with people, and maintaining distance. In these difficult and terrible circumstances, we were affected both physically and mentally. Unfortunately, the pandemic claimed numerous lives causing much pain and grief to many people and families, affecting to a lesser or greater extent our mental health.

Living in a confined environment that limited the ability to feel and express emotions reduced our emotional strength and constrained the possibility of positive emotions. In addition of the limited opportunities of social contact, our environment was in turn surrounded by expressionless faces covered with face masks. But despite all these limitations, our emotional resilience helped us to keep our mentally healthy to survive the pandemic. In the absence of physical contact, the option was to interact on the digital realm of internet and social networks. Although we had already used them, the exceptional situation of the pandemic caused us to make use of them even further, turning into the “normal”, and changing forever the feeling and expression of emotions.


A brave new “emotional” world.

Emotions are a physical adaptation and communication response of our body to different situations. We experience them daily: good, or bad, intense, or moderate, pleasant, or unpleasant. But nowadays more and more of our emotions occur both in the physical “offline” realm as well as in the digital “online” realm, up to the point that our emotions have become “hybrid”, coexisting in both worlds. While our traditional face-to-face social life occurs more slowly, located in a specific physical location, our social life in the digital realm occurs more rapidly, is omnipresent, and can occur beyond a physical location at the distance. In the digital realm, our expressed emotions are continuously recorded, creating a huge amount of stored data (big data), which in turn contribute without noticing it to the development and shaping of our inner subjectivity and personality.

Even though  there are differences and similarities between the expression of emotions in face-to-face relationships compared to the relationships mediated by digital technology, every day we tend to express ourselves more and more emotionally in one of the 3 main messaging applications WhatsApp, Facebook Messenger and WeChat, which combined add up to more than 4 billion monthly active users. We have become so used to expressing our emotional mood digitally that we already do it all the time without realizing it. Who has not done it?


Emotions, feelings, and sentiments

The understanding of our emotions, feelings, and sentiments in the digital realm of social networks and messaging applications is becoming very important for the psychological comprehension of our reality and mental health.  Without going too deeply, in a nutshell an emotion is a complex reaction pattern, involving experiential, behavioral, as well as physiological elements in response to a significant matter or event. There is a large set of emotions, but the set of primary emotions that are recognized universally across cultures are: fear, anger, joy, sadness, disgust, contempt, surprise, shame, shyness, and guilt.  Emotion typically also involves feeling which is any experienced sensation or a self-contained subjective phenomenal experience. In that sense a feeling is a personal, internal, and intimate experience. On the other hand, a sentiment is a mental attitude, a thought that has been influenced by emotion. A sentiment allows the individual to transmit her emotion through expression, with an action, which involves a relationship with an object or a person.

Photo by Gratisography from Pexels

Although emotions are recognized universally across cultures their physical expression varies slightly from person to person with facial movements such as smiling, frowning, crying, laughing, hugs, clapping, yelling, fast heartbeats. Furthermore, we are not always aware of these emotional expressions, we simply feel and express them, they arise automatically, spontaneously, and naturally.

It is beneficial to express our emotions in an appropriate way according to the place, and the circumstance. Not expressing properly our emotions, being emotionally dry, inexpressive, stern, is potentially harmful and can affect our mental health in the long term. Unexpressed emotions do not disappear, they remain in our unconscious without us realizing it, and they will emerge sooner or later, when the opportunity arises or when the pressure for them to reveal is too intense, giving rise to a distorted and unhealthy emotional expression.

Unexpressed emotions will never die. They are buried alive and will come forth later in uglier ways

Sigmund Freud

Emotion expression in the digital realm

We undoubtedly can identify and detect the emotions of ourselves and the people around us. It is an aptitude that we all possess since born by nature, which is also nurtured by the living experience in our society and culture. However, we nowadays need to express our emotions not only in the physical realm, but in a digital realm as well, with interactions mediated by technology, given the person on the other side cannot be in front of us face to face. We inadvertently were pushed to develop the skill to perceive and express our emotions by interpreting only the messages through the texts as we write. And fortunately, technology provide us a witty and effective way of doing it only with messages, while preserving the richness of the expression and interpretation of our emotions. Certainly, we all already know how to do it just by using: emojis, stickers and animated gifs.

Photo by Domingo Alvarez E from  Unsplash

 We all surely know what an emoji is, a pictogram embedded in text used in electronic messages. The emoji can remarkably well transmit our emotional cues 🙂 😉 😦 . Prior to emojis, the “emoticon” was the option used to express emotions with just text-based symbols such as : – ) (happy) and : – ( (sad). However, emoticons are no longer used, except in a few cases by fewer people. The sticker is a refined, detailed, and bigger emoji, an illustration of a funny moving character representing an emotion or action. The animated gif is another emotional expression figure, which has been steadily growing, especially among Gen Z group. It is usually available on all the messaging apps and popular internet libraries such as giphy, reddit and tenor. To see how big is the spectrum of emotions that can be expressed with emojis, just go to emojipedia, the world’s #1 emoji reference. To have a glimpse of an incredible view in real time of the astonishing penetration and usage of emojis on twitter just go to the emojitracker.

However, compared with real people faces, pictures or words, emoji is still ambiguous because it does not accurately symbolize a discrete emotion or a feeling mood. Thus, their meaning relies on the context of the message in which they are embedded. Fischer and Herbert C (202) research has shown that emoji is often misunderstood; in some cases, this misunderstanding is related to how the actual emoji design is interpreted by the viewer; in other cases, the emoji that was sent is not shown in the same way on the receiving side.  A further problem relates to technology because the emojis are normally encoded with numbers. If the author and the reader do not use the same software or operating system in their devices, the emoji code will not match up, so then they could probably visualize the emoji with a slightly different character.


Digitally emotion detection and identification

Is there a way to detect and identify emotions using an automatic computational tool?. Certainly yes. Practical results have been achieved during the last decades, both in the academic world and technology industry. Paul Ekman, an American psychologist who stands out as a pioneer in the study of emotions and their relationship with facial expressions, defined six basic emotions that are universal: happiness, sadness, anger, fear, surprise, and disgust, and later included by other psychologists in this list more emotions such as pride, excitement, embarrassment, contempt, embarrassment, etc. Ekman contributed with a significant update to the encoding of emotions in the Facial Action Coding System (FACS) system which is a database of compiled facial expressions. The FACS system is a computational tool that can categorize the physical expression of emotions, detects faces in videos, extracts the geometrical features of the faces, and then produces temporal profiles of each facial movement. Its usage is still controversial, but FACS has been used to investigate human emotional behavior and facial recognition for training, scientific research, and marketing. Just for entertainment an curiosity you can watch a fictional application of FACS on the Lie to Me show.

Another important computational tool is Sentiment Analysis (also known as opinion mining).  Sentiment analysis makes use of natural language processing, text analysis, computational linguistics, to automatically identify, extract, quantify, and study affective states and subjective information. Some notably use cases are in consumer opinion and consumer reviews in well-known services such as consumer review sites, video streaming, web search, e-commerce, and social networks. Most of our postings, reviews, opinions and “likes”, in the social networks, streaming platforms, and consumer reviews sites, reflect some level of sentiment (positive, negative, or neutral), which are then later classified and categorized to extract a specific sentiment. That constitutes the pillars of a whole industry to identify our sentiments and in turn influence our choices by presenting the options which match to our sentiments detected. The sentiment analysis industry is growing rapidly mostly because of the huge data available in the consumer review sites and social networks, that make it possible for many applications to provide valuable information to business, government, and media, about the people´s opinions, sentiments, and emotions.

From a more clinical psychological perspective it is particularly interesting that social media data could also be used for psychoanalysis to study human personality and emotions by side. The vast amount of data available from Twitter or Facebook, provide a new type of data consisting of a mixture of text, emojis, image and video information. Over the time, our personality spontaneously reveals from stored data, more than could be possibly revealed in a few therapy sessions. Every day we express ourselves and leave footprints of our emotions on the social networks, most of the time unconsciously because we do not think in advance what we say, we simply act on impulse. On that sense, from the psychoanalytic point of view, we unconsciously and spontaneously disclose a glance of ourselves in the social networks. Emmert-Streib, F., Yli-Harja, O., & Dehmer, M. (2019) research conclude that the data output of twitter users could be analyzed through the psychoanalytic literary criticism methodology, making it possible a computational psychology approach to study the average human personality and emotions just using social media data.


Artificial Emotions

Artificial intelligence is certainly the more relevant technology that is increasingly penetrating our daily lives. A plethora of “smart” devices are nowadays common at home: smart assistant, smart watch, smart phone, robot toys, smart vacuum cleaner, smart TVs and many more. Ok, I must admit they are not really “smart” or intelligent, but they imitate of being so in a witty manner, with a little help of our cognitive tendency to perceive them as such.  

From all of them, smart assistants stand out as the more emotionally capable. Gradually the new generations of smart assistants can detect a basic set of emotions in humans and in turn simulate the function of emotion to convincingly interact with people. This is an important step that results from the cognitive psychology and affective computing research and development, directed to create computational models of emotion that allow the expression of emotion in computational agents. For example, Alexa just recently expanded its style options to new accents and languages, incorporating human nuances of feeling and occasion. The latest Alexa algorithm analyzes how humans speak when feeling the appropriate emotion and generates in turn a variation of Alexa’s synthetic voice to mimic it.  

Another important research and development are going on in affective computing, to design of computational devices with natural emotional capabilities and compellingly simulating of emotions. Affective computing growth expectations round 28% in the health care, robotics assistants, patient monitoring devices, senior companionship, and mental health care applications.


Last Remarks

Change is a distinctive feature of our current world. Only at the end of the 2000s, just approximately 15 years ago, the first smartphones emerged alongside the advent of social networks and messaging which significantly changed the way we carry out our social relationships. From a positive perspective, these changes turned out to be an advantage for emotional expression at distance.

I remember it take me a while to adapt when my children began to send to my cell phone a bunch of smiling faces, and animated cartoons. But over time I get used to, and now, I like it because it’s fun and entertaining. Nowadays we normally write each other in the friends and family chat group, with emojis and animated gifs. This of course does not affect the fact that we keep gathering and frequent each other, while at the same time it has the advantage of keeping us emotionally connected at distance.

However, we must take care to avoid extremes, and not to fall into the unwanted usage of “technoference” or “phubbing”, which results from paying attention to the mobile phone rather than connecting to people face to face. The abuse of mobile phone, in the long term can become a detrimental trait that could affect the relationships with our loved ones.

The change pace will certainly continue at an accelerated rate with more coming on: artificial intelligence, smart assistants, robotics, enhanced social networks, and virtual worlds (metaverse). The emotional expression in the physical realm has been greatly expanded into the digital realm. We should adapt to this change in a positive and constructive way and be cautious that the technology not overpass our control. We indeed can affirm that our emotional realm is not exclusively physical anymore, it is now “hybrid”: the emotions expressing and spreading both physically and digitally.


References

Borod, J. C. (Ed.). (2000). The neuropsychology of emotion. Oxford University Press.

Carolus A, Wienrich C, Törke A, Friedel T, Schwietering C and Sperzel M. (2021). ‘Alexa, I feel for you!’ Observers’ Empathetic Reactions towards a Conversational Agent. Front. Comput. Sci. 3:682982. doi: 10.3389/fcomp.2021.682982

Choudhury, M.D. (2012). The Nature of Emotional Expression in social media: Measurement, Inference and Utility.  Technical Report, Microsoft.

Derks, D., Fischer, A.H., & Bos, A.E. (2008). The Role of Emotion in Computer-mediated Communication: A Review. Computers in Human Behavior, 24(3), 766-785. doi: 10.1016/j. -chb.2007.04.004

Derks, Daantje & Bos, Arjan & von Grumbkow, Jasper. (2008). Emoticons in Computer-Mediated Communication: Social Motives and Social Context. Cyberpsychology & behavior: the impact of the Internet, multimedia and virtual reality on behavior and society. 11. 99-101. doi: 10.1089/cpb.2007.9926.

Emmert-Streib, F., Yli-Harja, O., & Dehmer, M. (2019). Utilizing social media Data for Psychoanalysis to Study Human Personality. Frontiers in psychology, 10, 2596. doi: 10.3389/fpsyg.2019.02596

Fischer B and Herbert C (2021) Emoji as Affective Symbols: Affective Judgments of Emoji, Emoticons, and Human Faces Varying in Emotional Content. Front. Psychol. 12:645173. doi: 10.3389/fpsyg.2021.645173

Kharde, Vishal & Sonawane, Sheetal. (2016). Sentiment Analysis of Twitter Data: A Survey of Techniques. International Journal of Computer Applications. 139. 5-15. doi: 10.5120/ijca2016908625.

Liu, B. (2015) Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. Cambridge University Press, Cambridge.
https://doi.org/10.1017/CBO9781139084789

M.D. Devika, C. Sunitha, Amal Ganesh, (2016) Sentiment Analysis: A Comparative Study on Different Approaches, Procedia Computer Science, Volume 87, 2016, Pages 44-49, ISSN 1877-0509, doi: 10.1016/j.procs.2016.05.124.

Prikhodko, O. V., Cherdymova, E. I., Lopanova, E. V., Galchenko, N. A., Ikonnikov, A. I., Mechkovskaya, O. A., & Karamova, O. V. (2020). Ways of Expressing Emotions in Social Networks: Essential Features, Problems and Features of Manifestation in Internet Communication. Online Journal of Communication and Media Technologies, 10(2), e202010. doi: 10.29333/ojcmt/7931

Robinson Peter and el Kaliouby Rana (2009). Computation of emotions in man and machines. Phil. Trans. R. Soc. B 364: 3441–3447.  doi: 10.1098/rstb.2009.0198

Rúa-Hidalgo, I.; Galmes-Cerezo, M.; Cristofol-Rodríguez, C.; Aliagas, I. (2021. Understanding the Emotional Impact of GIFs on Instagram through Consumer Neuroscience. Behav. Sci. 2021, 11, 108.   doi: 10.3390/bs11080108

Stets, J.E. (2006). Emotions and Sentiments. In: Delamater, J. (eds) Handbook of Social Psychology. Handbooks of Sociology and Social Research. Springer, Boston, MA . doi: 10.1007/0-387-36921-X_13

Tiwari, Shudhanshu & Sinha, Anurag. (2020). Sentiment Analysis of Facebook Data using Machine Learning. International Journal of Innovative Research in Applied Sciences and Engineering. 4. 2456-8910. doi: 10.29027/IJIRASE.v4.i4.2020.735-742


Featured Photo by Kindel Media from Pexels

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s