Title Page

Contents

Abstract 9

Chapter 1. Introduction 10

1.1. Introduction 10

1.2. Overview of Research 11

1.3. Outlook of this Thesis 12

Chapter 2. Background 14

2.1. Neural Networks 14

2.1.1. Classification 15

2.1.2. Out-of-distribution detection 15

2.2. Literature Review 17

2.2.1. Out-of-distribution Detection 17

2.2.2. Discriminative Representation Learning 20

Chapter 3. Proposed Method 22

3.1. Margin Extension via Angular Margin Loss 22

3.2. Gently Sloped Margin via Weight Regularization 27

3.3. Out-of-distribution Detection 29

Chapter 4. Theoretical Analyses 30

4.1. Detection Error Minimzation 30

4.2. Overconfidence Relaxation 32

Chapter 5. Experiments 34

5.1. Preliminary Assessment 35

5.2. Comparative Experiment 37

5.3. Combination with Post-hoc Processes 40

Chapter 6. Discussions 41

6.1. Robustness to Hyperparameter Setting 41

6.2. Preservation of Classification Accuracy 42

6.3. Computational Efficiency 43

Chapter 7. Conclusion 45

References 46

Appendix 52

논문요약 54

Table 2.1. Comparison of OOD Detection Methods with and without Side-effects 19

Table 5.1. OOD Detection Comparison with a Baseline Method 35

Table 5.2. OOD Detection Performances 37

Table 5.3. OOD Detection Improvement by Combining with Post-hoc Approaches 40

Table 6.1. Detection & Classification Performance 43

Table 6.2. Elapsed Time Comparison 44

Figure 3.1. Visualization of features learned with (a) softmax loss and (b) angular margin loss. The upper row was implemented using a modified version of the LeNet-5 network with... 25

Figure 3.2. Cosine similarities for (a) intra-class and (b) inter-class at increasing number of epochs 26

Figure 3.3. Effect of weight regularization: margin contour between classes(two-class example for upper row and four-class example for lower row) and MSP value of an OOD... 28

Figure 5.1. Distributions of MSP values: (a) softmax loss, (b) softmax loss with weight regularization, (c) angular margin loss with weight regularization 35

Figure 6.1. AUROC performance of the proposed method at varying hyperparameters (a) varying m with fixed s (b) varying s with fixed m 42