Editor’s Note: Anastasios Economides explores the relationship between emotions and learning. There is a large body of research to determine reliable detection of emotions, and additional research that relates emotions and learning. This article parallels biofeedback in the way it uses affective feedback in testing to enhance learning efficiency and effectiveness.

Emotional Feedback in CAT
(Computer Adaptive Testing)

Anastasios A. Economides

Abstract:

It is well known that emotions influence learning. Also, feedback is a powerful educational tool which enhances learning. This paper aims to provide real-time adaptive feedback to emotionally support the examinee during Computer Adaptive Testing (CAT). It describes a framework for employing emotional feedback in CAT. After the CAT system recognizes the examinee’s current emotional state, it supports him emotionally using emotional feedback. The paper proposes several emotional feedback types classified with respect to the emotions and their time of triggering. Finally, the CAT system provides personalized emotional feedback to the examinee according to his current emotional state.

Keywords: adaptive feedback, affective computing, computer adaptive testing, emotions, feelings, personalized feedback.
 

Introduction

Vygotsky (1987) observed that the study of psychology had been damaged by the separation of the intellectual from the motivational and emotional (or affective) aspects of thinking. The terms emotions and affect refer to states such as happiness, shame, fear, disgust, annoyance, sadness, anger, equanimity, anxiety, depression, surprise, and love. It is well known that emotional upsets can interfere with mental activities. Students who are anxious, angry, or depressed do not learn (Goleman, 1995).

Our feelings can profoundly influence how we approach a reasoning task. For example, sadness may support analytical problem solving (Schwartz, 2002). Affective neuroscience and psychology have reported that human affect and emotional experience play a significant, and useful, role in human learning and decision making (Bechara at al., 1997). The influence of positive affect on creativity has been well demonstrated (Isen et al., 1987; Isen, 2001). The positive affect enhances problem solving and decision making, leading to cognitive processing that is not only flexible, innovative, and creative, but also thorough and efficient. Experiments indicated that positive affect, induced by means of seeing a few minutes of a comedy film or by means of receiving a small bag of candy, improved performance on tasks that are generally regarded as requiring creative ingenuity. A slight positive mood does not just make you feel a little better but also induces a different kind of thinking, characterized by a tendency toward greater creativity and flexibility in problem solving, as well as more efficiency and thoroughness in decision making (Isen, 2000).

Poor learning can produce negative emotions; negative emotions can impair learning; and positive emotions can contribute to learning achievement and vice versa. Induced negative emotions have been shown to hamper performance on cognitive tasks, whereas positive emotions have an opposite effect (Izard, 1984). Also, inducing a sad mood in very young children increased the time it took them to learn to respond to a task, and also increased their number of errors; inverse results were achieved by inducing happiness (Masters et al., 1979). Numerous studies have demonstrated a connection between anxiety and academic performance: the more anxious a person is the poorer his academic performance (Seipp, 1991). Preschool children in a positive mood mastered a shape discrimination task more quickly and with fewer errors than did children in an induced negative mood. Youngsters identified as at risk for school failure were found to complete mathematical problems significantly more accurately under induced positive-mood conditions (Tanis and Bryan, 1991).

However, it is not an easy task to recognize and measure emotions. There is evidence from the neuroscience about the close relationship between emotion and homeostasis (Damasio et al., 2000). Positive affect is associated with increased brain dopamine levels and influences olfaction, the consolidation of long-term (i.e. episodic) memories, working memory, and creative problem solving (Ashby at al., 1999). Fear, anger, sadness, and joy show up in the brain as different patterns of blood flow, providing one possible explanation for how affect influences brain activity. Different regions of the brain participate in happiness, sadness, and disgust (Lane at al., 1997).

Another line of research on recognizing and measuring emotions use sensors, such as body-worn accelerometers, rubber and fabric electrodes, miniature cameras and microphones, and garment or accessory-type devices, along with pattern recognition algorithms of facial expressions from video or/and vocal expressions or of stress patterns from thermal imagery of the face and other physiological measures (Picard, 1997).

Recognition (81% accuracy) of eight emotional states of a human given four physiological signals has been achieved (Picard at al., 2001). The Facial Action Coding System (FACS) is widely used in behavioral investigations of emotion, cognitive processes, and social interaction (Donato at al., 1999). Action units that include upper facial muscle movements such as inner eyebrow raise, eye widening have been used to achieve a recognition accuracy of 62.5% to 69.3% (Kapoor et al., 2003). Automated speech recognition achieved 50—60% accuracy on emotional speech (Steeneken and Hansen, 1999). Measuring the pressure on the mouse, steering wheel, etc. using force sensitive resistors shows also promising results (Mota and Picard, 2003, Dennerlein at al., 2003). Frustrated users applied higher force to the side of the mouse and the average wrist extensor muscle activity. Information from chair pressure patterns have been used (accuracy 82% to 98%) to recognize a child’s interest level (Kapor et al., 2001).

Combination of the visual and audio information for emotion detection and recognition has been proposed (DeSilva at al., 1997; Chen at al., 1998; Russell at al., 2003). Recognition accuracy of 67.8% to detect interest has been reported (Kapoor at al., 2004). Electrocardiogram, electromyogram, respiration and skin conductance sensors to measure autonomic nervous system activation have been embedded in an automobile to capture episodes of driver’s stress (Healey and Picard, 2000). The drivers’ stress was classified in four levels with 89% accuracy.

In this paper, we propose the use of emotional feedback in Computer Adaptive Testing (CAT). Developments in psychometric theories, computer and networking technologies have enabled the widespread development of CAT. In CAT, when the examinee answers correctly a question, then the next question is harder. Otherwise, if the examinee answers wrongly, then the next question is easier (Wainer at al., 2000; Sri Krishna, 2001). The rapidly increasing number of CAT systems is due to the main advantages that it offers. The examinee answers questions tailored around the level of his knowledge and abilities. Since the questions are not too hard for him, he is not discouraged. Also, since the questions are not too easy for him, he does not get bored and no time is wasted. Furthermore, each examinee sees different question each time he takes the test and also different questions from the other examinees. So, the cheating is restricted. Finally, the results are immediate, reliable and valid.

Feedback is an important factor in learning. The feedback aims at learner-centered education. It may help the examinee in the following ways:

• Trigger and stimulate his attention and focus.

• Activate his involvement and interaction with the test.

• Activate and develop his knowledge and abilities.

• Instruct and teach him.

• Guide him.

• Inform him about his progress, strengths, weaknesses, errors.

• Correct his misconceptions.

• Prevent his failures and mistakes.

Despite the many benefits that feedback can provide in learning, it has not been widely introduced into the CAT systems. Perhaps, this is because most CAT systems are exclusively used for formative assessment rather than for self-assessment and learning. In addition, the introduction of feedback into the CAT systems is not a simple process. It needs careful requirements analysis, planning, design, development, evaluation, and redesign.

Previous research considers that the feedback informs the examinee about the answers and the results (Kulhavy and Stock, 1989; Ross and Morrison, 1993; Mason and Bruning, 2001). However, it is important to also support the examinee emotionally. In this paper, we propose emotional feedback to enhance learning and problem solving. This is the first paper to investigate emotional feedback in CAT. We present various types of emotional feedback in CAT. The CAT system may sense and monitor the examinee (e.g. face and eyes expressions, voice tone, blood pressure, heartbeat, breath rhythm, temperature, brain waves, muscle tension, skin conductivity, etc.) and recognize his emotional states. Then, the CAT system may present to the examinee personalized emotional feedback according to the examinee’s current state. This real-time personalized emotional feedback tries to emotionally support the examinee.

Emotional Feedback types

Having documented the influence of emotions on learning, we proceed to frame the use of emotional feedback in CAT. The computer continually senses, measures and recognizes the examinee’s current state. At appropriate instances, it provides to the examinee personalized emotional feedback according to his current state. First, we classify the emotional feedback with respect to the emotion type. Then, we classify it with respect to the triggering instance.

In the classification with respect to the emotion type, we consider three emotional feedback categories:

1. Positive emotions feedback,

2. Control of Negative emotions feedback, and

3. Negative emotions feedback. Let first define these emotional feedback categories.

A. Positive emotions feedback: it acts and expresses positive emotions to the examinee trying to develop, maintain and increase his positive emotions. So, it may try to increase the following examinee’s emotions:

1. Enthusiasm, fascination, excitement, passion and involvement with the test.

2. Happiness, joy, delight, pleasure and amusement.

3. Satisfaction and fulfillment.

4. Calmness, tranquility, serenity, peacefulness, comfort and relaxation.

5. Hope and optimism.

6. Expectancy, anticipation, certainty, assurance and acceptance.

7. Sympathy and love.

8. Pride and honor.

B. Control of negative emotions feedback: it tries to control the examinee’s negative emotions. So, it may try to control the following examinee’s emotions:

1. Boredom and apathy.

2. Sadness, melancholy, sorrow and depression.

3. Anger, irritation, indignation and upset.

4. Anxiety, stress and nervousness.

5. Fear, concern, worry and doubt.

6. Confusion.

7. Pessimism, defeatism and self-pity.

8. Frustration, despair, hopelessness and panic.

9. Astonishment, amazement and negative surprise.

10. Shame, guilt, humiliation, embarrassment and dishonor.

11. Disgust and aversion.

12. Hate.

We further classify the Control of negative emotions feedback into 4 sub-categories:

a)      Avoidance and prevention of negative emotions feedback: it tries to avoid and prevent the development of negative emotions;

b)      Control and management of negative emotions feedback: it tries to control and manage the examinee’s negative emotions not allowing them to grow;

c)      Relief and alleviation of negative emotions feedback: it tries to relief and alleviate the examinee’s negative emotions lowering them;

d)      Transformation of negative emotions feedback: it tries to transform the examinee’s negative emotions to positive ones.

C. Negative emotions feedback: it expresses negative emotions to the examinee trying to increase his effort and commitment.

Next, we provide specific emotional feedback types:

1. Humor and Jokes. It amuses the examinee making him happy. It calms and cools down him. It relieves his sadness.

2. Entertainment and Games. It amuses and pleases the examinee.

3. Reward. It develops the examinee’s passion, expectancy, and optimism. It increases his satisfaction and honor. It alleviates his anger.

4. Sympathy and Goodwill. It increases the examinee’s delight, fulfillment, peacefulness, acceptance and sympathy. It decreases his anger and hate.

5. Positive Surprise. It increases the examinee’s excitement. It prevents his boredom and apathy.

6. Encouragement. It increases the examinee’s involvement, enthusiasm and hope. It decreases his doubt, pessimism and anxiety. It alleviates his frustration and disgust.

7. Acceptance. It increases the examinee’s assurance, calmness and fulfillment. It decreases his irritation and anxiety. It reduces his shame and guilt.

8. Praise and Congratulations. It increases the examinee’s assurance, fulfillment and pride. It decreases his worry, defeatism and shame.

9. Criticism. It challenges the examinee and increases his effort and commitment.

10. Punishment. It energizes the examinee and increases his effort and work.

Next, we present the classification with respect to the feedback activation instance.

Activation Instance of the Emotional Feedback

Depending on the appearance time of the emotional feedback, we classify the emotional feedback into the following categories:

1. In advance emotional feedback: it supports emotionally the examinee beforehand of an action. For example, it may enhance his enthusiasm, hope and optimism before the test start. It may reduce his stress; relax his anxiety and fear before presenting a question.

2. Immediate emotional feedback: it supports emotionally the examinee immediately after of an action. For example, it may comfort the examinee after the test start. It may reduce his panic after a wrong answer. It may encourage, praise and congratulate him on his effort, on his results, etc. However, it may also criticize and blame him for not studying enough after a wrong answer to an easy question.

3. Delayed emotional feedback: it supports emotionally the examinee after some time of an action. For example, it may try to cool down him after the test. Or, it may try to tranquilize an agitated examinee. It may congratulate or criticize him about his effort or results.

More specifically, for the emotional feedback in CAT, we distinguish: 1. Pre-Test emotional feedback, 2. Pre-Answer emotional feedback, 3. After-the-Answer emotional feedback, and 4. After-the-Test emotional feedback.

1. Pre-Test emotional feedback: it is presented to the examinee on his request, or on teacher’s request, or automatically based on the examinee’s current state before the test starts.

For example, it may excite the examinee about the test subject by presenting real life cases and future perspectives. It may inspire his curiosity asking introductory questions that trigger his imagination. It may challenge him by presenting to him the maximum score achieved by other students previously. It may create a pleased, friendly, comfort, joyful and cheerful environment. It may enhance his optimism about his success in the test by comparing the difficulty level of the test to other tests he succeeded in. It may make him aware of what to expect from the test, the questions, and the result interpretations. It may help his prediction, forecasting and foresight about the test questions (e.g. with respect to the question types, difficulty levels, subjects, format and media). It may prepare him to be ready for the questions by presenting sample questions. It may increase his certainty and surety that the questions will be on the subjects that he had been taught and studied. It may increase his belief on the test usefulness and meaningfulness by showing real life applications. It may increase his trust and confidence on the test credibility by showing reliability and validity analysis of the questions. It may relax his concerns about the test fairness by showing statistical results from previous test with respect to discrimination factors (e.g. gender, ethnicity). It may reduce his disgust about the test subject by showing to him its practicality and usefulness.

2. Pre-Answer emotional feedback: it is presented to the examinee on his request, or on teacher’s request, or automatically based on the examinee’s current state after a question is presented to him and before he answers it.

For example, it may energize and stimulate the examinee by asking to make preliminary selections. It may enjoy and amuse him by providing jokes, cartoons, music, etc. It may encourage him by confirming that he can answer correctly. It may assure him that the question difficulty and subject are appropriate for him. It may clarify what the question is asking for. It may assure him that there are not tricks and hidden traps in the question. It may motivate him about the importance and meaningfulness of the question. It may increase his confidence and belief that he will answer correctly. It may show understanding, empathy and compassion to him about the question difficulty. It may lessen his shame for not understanding the question by showing that other examinees could not also understand the question. It may prepare him for the difficulty and the subject of the next question. It may assure him on the appropriateness and suitability of the next question.

3. After-the-Answer emotional feedback: it is presented to the examinee on his request, or on teacher’s request, or automatically based on the examinee’s current state after he answers the question.

For example, it may excite it may reward the examinee with entertainment for his correct answer. It may congratulate and praise him for answering correctly. It may feel for and show compassion to him for answering wrongly. It may excuse him for his wrong answer. It may criticize and chide him for his wrong answer. It may allay and appease his irritation and upset in case he does not agree with the question result. It may justify the correct answer in case he strongly disagrees. It may cool down him with music after his strenuous effort to answer the question. It may confirm and assure him that he is doing well. It may assure him that one wrong answer is not a disaster. It may explain to him the importance of his answer in the final score. It may assure him on the right contribution of his answer to his overall score.

4. After-the-Test emotional feedback: it is presented to the examinee on his request, or on teacher’s request, or automatically based on the examinee’s current state after the test end.

It may agree with his satisfaction with respect to his effort and his performance. It may support his happiness and joy for succeeding the test. It may honor and flatter him by admiring his performance. It may show understanding and excuse him for failing the test. It may show to him understanding about the test difficulty. It may show to him mercy for failing the test by letting him to take it again. It may alleviate his anger towards himself for failing questions that he knew the correct answer. It may assure that everything is done properly and the result interpretation will be fair. It may assure that this test is not everything in life and that there are also other chances to take it again. It may remove his concerns and doubts about the accurate interpretation of the test results. It may relax his worry about the fair and confidential use of the test results. It may prepare him for what to expect with respect to the test results interpretation and use. It may assure him of the reliability and validity of the test results interpretation. It may positively surprise him by giving to him extra points for “good” behavior (e.g. short answer times, minimum resource usage). It may recognize his effort. It may decrease his hate about tests by showing to him the benefits from the test.

Conclusions

Feedback is an important mechanism in learning. Previous research in computer testing considers only feedback that informs the examinee about the answer to every question. This paper proposes personalized emotional feedback in CAT to emotionally support each individual examinee. It also presents classifications of the emotional feedback according to the emotions induced, as well as according to the triggering instances. The CAT system may present emotional feedback to the examinee before the test start, after the question presentation and before his answer, after his answer, and after the test end. Based on the examinee’s current state, the most appropriate emotional feedback type should be invoked. For example, if the examinee answers wrongly, a hint or an alternative question version with encouragement comments may help him. It may show understanding to the examinee by agreeing on the question difficulty. It may assure the examinee that he is on the right track and he is doing well in the exam. It may attract his attention and inspire his curiosity about the test. It may challenge him. It may motivate him about the test educational objectives, and the obtained benefits.

This is the first attempt to introduce emotional feedback in CAT. As such, it provides the framework on which further practical cases will investigate the actual improvement on examinee’s performance and satisfaction that is achieved using personalized emotional feedback. It may stimulate future research on specific feedback constructs to generate specific examinee’s emotions. The presented ideas may trigger further investigation towards emotions supported learning. Designers and developers of CAT systems may include such personalized emotional feedback into their CAT systems. Then, evaluation of real systems will investigate the emotional feedback effect on the examinee’s satisfaction and performance in real exams.

References

Ashby F.G, Isen, A.M. and Turken, U. (1999), “A neuropsychological theory of positive affect and its influence on cognition”, Psychological Review, 106, No 3,
pp. 529—550.

Bechara, A., Damasio, H., Tranel, D. and Damasio, A. (1997), “Deciding Advantageously Before Knowing the Advantageous Strategy”, Science, 275, pp. 1293-1295.

Chen, L.S., Huang, T.S., Miyasato, T. and Nakatsu, R. (1998), “Multimodal human emotion/expression recognition”, In: Proceedings of International Conference on Automatic Face and Gesture Recognition, Nara, Japan, IEEE Computer Society.

Damasio, A.R., Grabowski, T.J., Bechara, A., Damasio, H., Ponto, L.L.B., Parvizi, J. and Hichwa, R.D. (2000), “Subcortical and cortical brain activity during the feeling of self-generated emotions”, Nature Neuroscience, Vol. 3, No 10, pp 1049—1056.

Dennerlein, J., Becker, T., Johnson, P., Reynolds, C. and Picard, R.W. (2003), “Frustrating Computer Users Increases Exposure to Physical Factors”, in Proceedings of the International Ergonomics Association, Seoul, Korea.

DeSilva, L.C., Miyasato, T. and Nakatsu, R. (1997), “Facial Emotion Recognition using Multi-modal Information”, in Proceedings of IEEE International Conference on Information, Communications, and Signal Processing, Singapore.

Donato, G., Bartlett, S., Hager, M., Ekman, J.C.P. and Seinowski, T. (1999), “Classifying facial actions”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, No 10, pp. 974—989.

Goleman, D. (1995), “Emotional Intelligence”, Bantam Books, New York.

Healey, J. and Picard, R.W. (2000), “SmartCar: detecting driver stress”, in Proceedings of International Conference on Pattern Recognition (ICPR'00), pp. 4218-4221, September 3-8, 2000, Barcelona, Spain, IEEE Computer Society.

Isen, A. M., Daubman, K. and Nowicki, G. P. (1987), “Positive affect facilitates creative problem solving”, Journal of Personality and Social Psychology, Vol. 52, pp. 1122-1131.

Isen, A.M. (2000), “Positive affect and decision making”, in Lewis, M. and Haviland, J. (Eds): “Handbook of emotions”, Guilford, New York.

Isen, A.M. (2001), “An Influence of Positive Affect on Decision Making in Complex Situations: Theoretical Issues With Practical Implications”, Journal of Consumer Psychology, 11(2), 75-85.

Izard, C.E. (1984), “Emotion-cognition relationships and human development”, in C.E. Izard, J. Kagan, & R.B. Zajonc (eds.), Emotions, cognition, and behavior, Cambridge University Press, New York.

Kapoor, A., Qi, Y. and Picard, R.W. (2003), “Fully Automatic Upper Facial Action Recognition”, in Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures (AMFG 2003) held in conjunction with ICCV 2003, pp. 195-202, Nice, France.

Kapoor, A, Picard, R.W. and Ivanov, Y. (2004), “Probabilistic combination of multiple modalities to detect interest”, in Proceedings of the International Conference on Pattern Recognition, Vol. III, pp. 969- 975, Cambridge, England.

Kulhavy, R. W. and Stock, W. A. (1989). “Feedback in written instruction: The place of response certitude”, Educational Psychology Review, Vol. 1, No 4, pp. 279 – 308.

Lane, R., Reiman, E.M., Ahern, G.L., Schwartz, G.E. and Davidson, R.J. (1997), “Neuroanatomical correlates of happiness, sadness and disgust”, American Journal of Psychiatry, 154, pp. 926—933.

Mason, B.J. and Bruning, R. (2001), “Providing feedback in computer-based instruction: What the research tells us”, http://dwb.unl.edu/Edit/MB/MasonBruning.html

Masters, J.C., Barden, R.C., & Ford, M.E. (1979). “Affective states, expressive behavior, and learning in children”, Journal of Personality and Social Psychology, 37, pp. 380-390.

Mota, S. and Picard, R.W. (2003), “Automated Posture Analysis for Detecting Learner's Interest Level”, Workshop on Computer Vision and Pattern Recognition for Human-Computer Interaction, CVPR HCI.

Picard, R.W. (1997), “Affective Computing”, The MIT Press, Cambridge, MA.

Picard, R.W., Vyzas, E. and Healey, J. (2001), “Toward Machine Emotional Intelligence: Analysis of Affective Physiological State”, IEEE Transactions Pattern Analysis and Machine Intelligence, Vol. 23, No. 10, pp. 1175- 1191.

Russell, J.A., Bachorowski, J.A. and Fernández-Dols, J.M. (2003), “Facial and vocal expressions of emotion”, Annual Review of Psychology, Vol. 54, pp. 329-349.

Schwarz, N. (2002), “Situated Cognition and the Wisdom of Feelings: Cognitive Tuning”, in Feldman Barrett, L. and Salovey, P. (Eds.) The Wisdom in Feeling, Guilford Press, New York, pp. 144-166.

Seipp, B. (1991), “Anxiety and academic performance: A meta-analysis of findings”, Anxiety Research, 4, pp. 27-41.

Sri Krishna, T.M. (2001), “Web-based adaptive testing”, Learning Technology newsletter, Vol. 3, No 3, pp. 44-47.

Steeneken, H.J.M. and Hansen, J.H.L., (1999), “Speech under stress conditions: Overview of the effect on speech production and of system performance”, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. IV, pp. 2079-2082, Phoenix, Arizona.

Tanis, B., & Bryan, J. (1991), “Positive mood and math performance”, Journal of Learning Disabilities, 24, pp. 490-494.

Vygotsky, L.S. (1987), The collected works of L.S. Vygotsky, Plenum, New York.

Wainer, H. et al. (2000), Computer Adaptive Testing: A Primer, Second Edition, Lawrence Erlbaum Associates, Inc., London.
 

About the Author

Anastasios A. Economides is Professor in the Information Systems Department, University of Macedonia, Thessaloniki, Greece.

Email: economid@uom.gr.