Title Page

Abstract

Contents

I. Background of emotional effect on in-vehicle cognitive tasks 15

1.1. Background of emotion as a potential distractive factor 15

1.2. Objective of study 16

II. Previous studies on distraction in driving 18

2.1. Factors influencing distracted driving 18

2.2. Adaptive behavior to distraction 19

III. Previous studies on measurement of cognition and emotion 20

3.1. Definition of emotion 20

3.2. Mental workload measurement 21

3.3. Visual perception and information processing 21

3.4. Interaction between emotion and cognition 22

3.5. Emotion and driving performance 24

3.6. Emotion elicitation and measurement 25

3.7. Design considerations of in-vehicle display 27

IV. Experiment for measurement of emotion effects in visual search performance 31

4.1. Motivation of studying emotional effect on visual search 31

4.2. Measurement of emotion and visual search performance 31

4.2.1. Experimental design – subject 31

4.2.2. Measurement of emotion and related apparatus 32

4.2.3. Experimental procedure 34

4.2.4. Analysis of EEG and visual search performance 35

4.3. Results of emotion effects on visual search 37

4.3.1. Visual search performance 37

4.3.2. EEG signals observed in visual search tasks 38

4.4. Discussion 40

V. Experiment for measuring effects of emotion on information acquisition 42

5.1. Emphasis of emotion effect on visual sampling of in-vehicle information 42

5.2. Experimental design for measurement of emotional impact on visual sampling 42

5.2.1. Selection of emotion-inducing videos 42

5.2.2. Experimental design – subjects 45

5.2.3. Experimental apparatus – in-vehicle instrument cluster display 45

5.2.4. Measurement procedure 46

5.2.5. Data analysis of EEG and visual sampling performance 47

5.3. Findings of emotion effects on in-vehicle display information acquisition 48

5.4. Correlations between EEG feature and driving related emotion induction 50

5.5. Discussion 52

VI. Conclusion and contribution 54

References 56

Appendix 65

Appendix A. Ergonomic design approach for external part. 65

Appendix B. Ergonomic system for vehicle maintenance 90

Table 3-1. ANS responses to emotions from previous studies 26

Table 3-2. Background/symbol color combinations suggested by ISO 28

Table 3-3. Emotional semantics and their matching design elements 30

Table 4-1. Summary of the four-factor (3 difficulty levels x 3 arousal level x 3 valence level x 2... 38

Table 4-2. P3 amplitude averaged between 300msec pre- and 100msec post-response over... 39

Table 5-1. Mean (SD) valence and arousal scores for driving-related video sample candidates 44

Table 5-2. Mean (SD) percentage of correct answers of in-vehicle information visual sampling 49

Table 5-3. Summary of the three-factor (3 difficulty levels x 3 arousal level x 3 valence level x 2... 50

Table 5-4. Mean (SD) subjective rating scores (Self-Assessment Manikin) to emotion eliciting... 51

Table A-1. Participants' information 66

Table A-2. Anthropometric measurements 66

Table A-3. Vehicle design parameters and their levels for each stature group 71

Table A-4. Body part discomfort questionnaires for specific time frame 72

Table A-5. Mean (SD) discomfort rating scores 76

Table A-6. Correlation analysis among discomfort rating scores 78

Table A-7. Mean errors of the approximation curve 80

Table A-8. The selected joints from trajectories data of short, middle, and tall stature group 82

Table A-9. Comparisons of model performances among different stature groups 86

Table A-10. Performances comparisons of models created by using different variables 87

Table B-1. Comparison between manual and automated gun barrel cleaning methods 94

Table B-2. Comparison of postural risks associated with the manual and automated gun barrel... 97

Table B-3. Comparison of cleaning quality achieved by manual and automated bore cleaning... 98

Figure 4-1. Emotional picture categories defined by valence and arousal. Grey dots represent... 32

Figure 4-2. Examples of visual search stimuli for each difficulty level. A shows target-absent... 34

Figure 4-3. Experimental procedure 35

Figure 4-4. Placement of 21 Electrodes following the international 10/20 system. A1, A2 are the... 36

Figure 4-5. The mean accuracy and reaction time with standard error in the case of A. target... 37

Figure 4-6. Grand mean ERPs, averaged over parietal electrode sites (P3/4) by target present... 39

Figure 4-7. Grand mean ERPs, averaged over parietal electrode sites (P3/4) around search task... 40

Figure 5-1. Mean rating scores of emotion eliciting video clips on valence and arousal... 44

Figure 5-2. Two types of instrumental cluster panels used in the experiment: (A) analog type,... 45

Figure 5-3. Emotion map 46

Figure 5-4. Mean percentage of correct recall of analog type vehicle instrument cluster... 48

Figure 5-5. Mean percentage of correct recall of digital type vehicle instrument cluster... 49

Figure 5-6. Averaged correlation coefficients for all 15 subjects and sequences. 51

Figure A-1. Anthropometric measurements 67

Figure A-2. Mock-up cabin used in the test 68

Figure A-3. Reflective marker placement locations 69

Figure A-4. Comparisons of geometric configurations of commercial vehicles in terms of four... 69

Figure A-5. Dimensional combination between the H30 and the floor height of the commercial... 70

Figure A-6. Ingress motion and the collected motion trajectories 71

Figure A-7. Illustration of Support Vector Machine 75

Figure A-8. (A) Distribution of discomfort score and (B) Distributional change after binary... 77

Figure A-9. Distribution of discomfort and distributional change after binary transform for (A)... 77

Figure A-10. Whole body ingress discomfort rating score and hip discomfort score on time... 78

Figure A-11. Comparison between original curves and normalized curve (vertical direction of... 79

Figure A-12. Comparisons between basic spline function and original trajectory for the vertical... 80

Figure A-13. Joints selected from filtering method. Bold line in the middle of the chart... 81

Figure A-14. The input joint selections for (A) short, (B) middle and (C) tall stature group 83

Figure A-15. Prediction accuracy changes as joint groups are introduced one at a time. As a... 84

Figure A-16. Step-wised joints input and corresponding change in prediction accuracy of SVM... 85

Figure B-1. comparison between (A) conventional bore cleaning poll and (B) newly developed... 94

Figure B-2. Working posture comparison between (A) manual bore cleaning method and (B)... 96

 The objective of this thesis was to examine if individuals' emotional state influences their visual search performance, with an ultimate goal to apply relevant findings to in-vehicle interaction designs. Two studies were conducted for this objective. The first study was to examine if induced emotions affect visual search performance, and the second study was to examine the effects of emotion on in-vehicle visual sampling performance.

In the first study, nine emotional states were defined by changing the levels of valence(negative/neutral/positive) and arousal (low/neutral/high). Emotions were induced using the International Affective Picture System (IAPS). Electroencephalograms (EEG) were recorded to find whether EEG data account for emotion and whether emotion affects cognitive performance. A total of 15 younger individuals (7 male / 8 female) completed two sessions.

In the first session, each individual rated 60 emotional pictures in terms of valence and arousal on the Self-Assessment Manikin (SAM). In the second session, each individual performed target search tasks after being exposed to emotional pictures. Task difficulty (low/medium/high) was manipulated by changing the number (8 or 16), shape (two types), or color (two types) of circularly arrayed visual stimuli. In order to assess top-town and bottom-up cognitive processes, target-present and target-absent stimuli were tested at each difficulty level. In each session, 21-channel EEG data were collected.

The effects of valence, arousal, and task difficulty on accuracy, reaction time, and ERP P3 components over the parietal area were all significant (p ≤ 0.046). Search accuracy was highest in a neutral-arousal, positive-valence state, while reaction time was shortest in a neutral-arousal, negativevalence state. ERP P3 components increased as accuracy increased and reaction time decreased. From the results of this study, it can be inferred that individual motivational experiences are connected to cognitive performances as well as inhibition and facilitation of neural activities.

In the second study, a total of 15 younger individuals (6 male / 9 female) participated. 30-s driving-related emotional video clips were shown to the participants and two types of cluster images (digital/analog) were randomly presented. The in-vehicle information presentation durations were three levels (0.4s / 0.8s / 1.2s). Driving-related emotional video clips were intended to induce five emotional states (arousal: 2 levels (high, low); valence: 2 levels (high, low); neutral on both arousal and valence). Participants were asked to recall the specific information (speed, revolutions per minute (RPM), fuel level, gear position and oil temperature) included in the cluster image.

Recall accuracy was better with the digital type cluster and the in-vehicle information visual sampling task performance was best under a neutral state of emotion. EEG signals were used to find the relationship between emotional ratings and psychophysiological signals. Theta (4-8Hz), alpha (8-12Hz), beta (12-30Hz), and gamma (30Hz~) frequency power spectrum features were extracted over 21 electrodes, and 1-s EEG signals with 50% overlapping time series of power spectrum features were analyzed. Alpha band asymmetry on frontal cortex, and theta band activation on negative emotion were observed in the valence dimension. In terms of valence, correlations of theta, alpha, and beta band with subjective ratings were weak. The left posterior-parietal site of gamma band power was highly correlated with the arousal score.

Major results of this study will be the improvement of Human-Machine Interface (HMI) by providing visual information adaptive to driver status and considering their cognitive and affective factors. Application area will be HMIs for automobiles and other transportation systems. In addition, this dissertation will contribute to the technological competitiveness of human interfaces in automotive and electronics industries, the competitive advantage in Intelligent Transportation Systems (ITS) vehicle development, the improvement of the ergonomic visual display evaluation method, and the acquisition of ergonomic visual display design principles. It is also expected to improve ergonomic evaluations of in-vehicle visual displays including cluster gauge and Head-Up Display (HUD). In addition, the result of this thesis can be utilized as fundamentals to construct cognitive, perceptual, and affective human models which are needed for the virtual evaluation of vehicle design involving digital human modeling tools.