Psycholinguistics/Prosody

Introduction

Emotion is a very important part of our conveyance and understanding of language. Research has shown that our understanding and memory of sentences and speech can be altered by emotional aspects. Emotions in speech can be conveyed from the words that we say or by the way that they are said. This idea works on a number of levels. For instance neutral sentences such as “I wanted cake” could have different meanings depending on the tone of voice of the speaker. Further words with seemingly positive or negative attributes can also be used with different tones to convey different meaning. The word ‘love’ typically has a positive emotional bias behind it, however if used in a sarcastic tone it can have negative meanings.

This idea is further explained in the 2008 study by Nygaard and Queen in which they state that listeners of speech not only obtain information from a speaker’s words but also the “non-linguistic properties of the utterances.” In similar terms Nygaard and Queen state that to understand completely the message conveyed by a speaker, the listener must understand what the speaker is saying (meaning of words and phrases being expressed) and how the speaker is saying it (most frequently classified through tone of voice). In this chapter we explore the aspects of emotional speech and comprehension to better understand the use and importance of emotion in language.

Expression of Emotion

To begin our discussion of emotion in language we will start with a discussion of the emotional aspects of speech. When discussing the expression of emotion an important starting point is prosody. Prosody refers to a wide range of ideas in language. In one sense it can be considered an aspect of the speech with regards to the speech cues such as pitch, loudness and so on or it could refer to the perception and understanding of the emotional cues that a speaker is trying to portray to a listener. Here we will refer to these two concepts as affective prosody and vocal emotion, which further explores the expression and recognition of emotion in speech.

Before we dive into our discussion of emotion in speech through the use of verbal cues, we should recognize that it is not the only way to allow for the expression of emotion in speech. As Bostanov and Kotchoubey explain their 2004 study on the recognition of affective prosody:

"In summary, there are three important features of nonverbal emotional exclamations. First, they convey the speaker’s affective state much better than words. Second, they most probably abstract some key prosodic features, primarily the voice quality, of verbal emotional speech. Third, the underlying physiological mechanisms are similar in all primates"

This is a great explanation of the different ways that emotional aspects are present in speech. However in our discussion we will stick to the verbal cues of emotion and their importance rather than the aspects of nonverbal emotional expression.

Affective Prosody

Our discussion of emotional speech will start with affective prosody. Affective prosody is the “suprasegmental aspects of speech that contain emotional as well as linguistic information.” To further explain this concept we can say that affective prosody relates to the prosodic cues and features of speech including pitch, loudness and articulatory rates . In a sense this relates heavily to how sentences are expressed verbally compared to graphically. In speaking, a listener cannot rely on such cues as periods, commas, question marks or other pieces of punctuation to help them know when people have finished one thought (sentence or phrase) and move on to another. This therefore has to be conveyed through the use of pauses in speech, stress on certain words and other verbal cues.

As stated by Cultler, Dahan and van Donselaar, prosodic structure has been noted as an important aspect in the recognition of speech. Therefore it is no wonder that it becomes important when discussing the idea of emotional conveyance in speech. In the sense of emotional speech, prosody allows for cues that allow us to stress the importance of particular words and phrase that represent our emotions certain topics. Reading a sentence such as I can’t believe he did that, will have very different meanings depending on where pauses and emphasis is placed. For instance if you were to pause after the word "he" compared to the word "that" the sentence would change in meaning from that idea that you are surprised at him compared to you are surprised at his action.

There is more to affective prosody than simply pauses and stress of certain words. Affective prosody also includes aspects of tone, pitch and loudness of speech. These are all important aspects that convey further meaning of speech to listeners. It is importantly noted here by Dupuis and Pichora-Fuller that even when using seemingly positive or negative words (positive words include compassion, love while negative words include hatred, anger) these words can have mixed meaning when used in different ways, such as sarcastic remarks.

Vocal Emotions

Our second topic in the expression of emotional language is that of vocal expression of emotion. This idea refers to not only the speaker’s expressions but also the reception and understanding of these ideas by a listener. In a sense, vocal expression of emotion does not relate to the prosodic cues and ideas of tone, loudness and articulatory rates that we discussed with affective prosody, but instead it focuses on the aspects of speech that relate to purely the emotional side of speech without any effect on the meaning of the words being conveyed.

When discussing the vocal expression of emotion, it is the interaction between speakers and listeners that needs to be carefully observed. It is this relationship that makes the expression of emotion important. This is because expression of emotion is based more off of understanding the speaker’s emotional state than actually understanding what they are saying. This idea becomes important when considering the theory that we base our understanding of what we are hearing from the emotional state of the person who is speaking to us. In the study done by Nygaar and Queen, not only was it found that emotional state of the speaker relates to the understanding of what is being said, but also mixed emotions between state and meaning of words can cause increases in the time it takes for a participant to understand the ideas being conveyed to them. In other words, when a speakers emotional expression was not the same as what they were saying (they were sad but were telling a happy story) the participants would take longer to understand what they speaker was saying.

Due to the great importance of understanding that comes from telling the emotion state of a speaker it is sometimes a mystery that humans can typically understand what is being said to them. However this is explained in a 2004 study by Bostanov and Kotchoubev in which they found that participants could identify all the emotions of speakers correctly with the primary use of voice quality. Further, although not overly important to this discussion, was the concept that of the nonverbal cues mentioned early. These are another important aid that can increase the understanding of a speaker’s emotional state.

To summarize emotional expression it is important to note the importance of affective prosody and vocal expression of emotion when trying to understand the information being provided to us by a speaker. It is also an important idea to note that these concepts occur simultaneously over the course of a conversation and are not aspects of speech that we have to think about turning on one while turning off the other.

Comprehension

The conveying of ideas is a completely useless tool without being able to understand what is being told to us. We will find in this section that there are many forms of comprehension that help us in the understanding of language. Even though there is more that can be discussed, for this section we will consider sentence comprehension to have two main types; emotional comprehension and non-emotional comprehension. By using these categories we will easily be able to distinguish between these two topics in this section.

Non Emotional Comprehension

To consider the effects of emotion on the comprehension of language we first must explore the ways in which language is typically comprehended outside of emotion. The two main ways non-emotional memory are verbatim memory and the gist effect.

Verbatim memory is the ability to perfectly recall a sentence that has been told or read by a person. This form of memory has been found to be less important than other types. This is because the use of knowing exactly what has been said is not necessary. In general the exact wording of what has been said is not that important. What is important is for humans to know the meaning conveyed in the sentence that they have just seen or heard. Imagine if whenever you were asked a question you could only give the answer within the context that you learned the answer. This would cause two problems, one you would take a longer to answer all questions and information would only be learned in certain contexts. Instead humans generally remember the main ideas and important aspects of things. This is called gist memory and it is the type of memory that focuses on the understanding of the meaning of the sentence that has been said, but not the exact wording.

The importance of these memory types has been explored by Sachs in the 1967 study which tested the ability of participants to remember the words of a sentence over time. What was found was participants could remember the meaning of the statement or gist memory, but could not remember the exact words that made up the statement over time. For instance if you read "The dog caught the ball in the park" you will be able to remember in a few minutes that the dog caught the ball but you will not remember for certain if you read that "The ball was caught by the dog in the park" or the first version of the sentence. The important thing to note is that verbatim memory is not overly important and as long as you know what was said being able to recite the exact sentence back is not normally necessary.

Emotional Attachment

Emotional memory refers to memory that is bolstered by emotional reaction to a certain topic. However, how far does this idea go? Can we understand and remember emotional ideas better than non-emotional ones? Is it possible to convey important ideas with emotional aspects to allow ourselves to remember them better? And are certain emotions better for memory than others?

While not all of these ideas will be answered in this section we can begin by saying that in general, emotional involvement in language makes the information being conveyed more memorable to a recipient . That is to say that when comparing speech that has no emotional connection to speech with emotional connection, the research has shown that emotional memory has a more lasting impression on the person obtaining the speech.

One study in which this is seen is the Kintsch and Bates study of 1977. In this study participants were able to recognize “extraneous sentences” such as jokes or other off the topic comments better than they could recognize the “descriptive statements” of a lecture . In this case both types of statements had a semantic meaning and were based off of gist memory rather than verbatim causing these issues to not matter. Therefore, this study is a great example of how emotionally charged statements (in this case with a positive, happy emotion) are better remembered than other semantically based statements. Furthermore, other research has agreed that language with a greater degree of prosodic aspects have an influence on the amount of linguistic processing that will occur .

Aprosodia

While it may seem automatic to be able to detect emotion in the context of a person’s speech, this is not an ability held by all people. Aprosodia is the inability to comprehend emotional cues in language. Further people with aprosodia also have do not have the ability to produce emotional aspects of language, such as the prosodic cues that we have discussed earlier. Aprosodia tends to be caused by damage to a variety of brain areas. While areas of the right hemisphere, especially to the anterior frontal lobe, are more widely recognized and studied, there is also evidence of damage to the homologous left cerebral structure, Broca’s area, causing aprosodia.

Research has been occurring in the last few years in the area of treatment for people with aprosodia. Even though there does not exist a set treatment at this time, there are a few treatments that are being tested. Two interesting techniques are the motoric-imitative and the cognitive-linguistic treatments. These treatments are based off two different causes for aprosodia. Motoric-imitative treatment is based from the concept that aprosodia is caused due to a motor impairment of the person effected. Therefore the treatment gets patients to say emotional statements (i.e “We won the game” [Happy] and “He was hit by a car” [Sad]) with the proper prosodic cues in the sentences. During the treatment help from the clinician would slowly lower as the patient became more comfortable using the emotional cues. In cognitive-linguistic treatment the patient learns the characteristics of different emotions to learn how they are expressed. This is done because of the belief that aprosodia is based from a decreased access to emotional words and cues by the patient.

Summary

Overall the importance of this chapter is to show the effect of emotion in language understanding and conveyance. In the end it should be recognized that emotion has an effect on the way in which language is portrayed to those around us both in the meaning of what is being conveyed and the memory processing that it will obtain.

Learning Exercise

1) Our first section of this chapter dealt with the vocal expression of emotion. One way it was said to be done was with prosodic cues that relate to a variety of speech features including pitch, articulatory rate, etc. Here we are going to explore these features and their ability to alter the meaning of sentences. For this exercise you will be asked to say the sentence “I have a test tomorrow that I have been studying all day for.” Each time you see the sentence different words will have either a 't' or 'p' written after it in brackets. When a word has a 't' after it then there is a change in the tone of the speaker and when a 'p' is written it means that there is a pause after that word. Say each sentence out loud and think about the change in meaning each sentence can have.

a. I have a test tomorrow that I have been studying all day for.

b. I have a test tomorrow (t,p) that I have been studying all day for.

c. I have a test (t,p) tomorrow that I have been studying all day for.

d. I have a test tomorrow that I have been studying (p) all day for.

e. I have a test tomorrow that I have been studying all day (t) for.

As you read through the sentence you should have noticed a slight change in meaning for each one. For instance during sentence (b) the change of tone in the word ‘tomorrow’ would lead to the thought that the speaker is worried about the test being so soon. In sentence (c) however, the change of tone at the word ‘test’ would seem that the person is always worried by tests.

2)The inability to detect emotion, aprosodia, has been discussed in this chapter but at times it can be hard to imagine. One of the best ways to understand this idea is to hear it. There exist many computer programs now that can read text, one of these will be needed for the following question. The useful part of these programs is that they are not good at conveying prosody. Therefore we can get an insight into what patients with aprosodia hear. Below are some sample sentences that can be used that tend to be conveyed with much emotion try putting them into a text to speech program or use whatever you want. Take not of the lack of prosody and how the meaning of the sentence isn’t as easily conveyed.

a. Your sister just had a baby.

b. John just landed in France.

c. I just finished writing my test.

d. Yesterday we went to the park, when we were met by a group of dogs.

You should notice that some of these sentences do not have a clear meaning without the proper emotion cues. Sentence (c) may be stating that you are happy to have finished a test, but it may also convey the fact that you did badly on it if you have negative cues in your speech. Sentence (d) may be happy in the fact that you met a group of fun dogs at the park, or it could be angered or sad if the dogs were bad behaving. Again try this experiment with other sentences that may have different meanings.

Answers to the following question can be found in this chapter:

1) Name the three types of memory from this chapter and state the differences between them.

2) What is aprosodia? Which parts of the brain are typically damaged?

3) What are the techniques used for patients with aprosodia?

4) What are the differences between verbatim and gist memory? Which one is used more in our day-to-day life?

5) State the different features of affective prosody.

6) Differentiate between affective prosody and the vocal expression of emotion.