Title Page
Contents
List of Abbreviations 14
ABSTRACT 15
Chapter Ⅰ. Introduction 18
1.1. Motivation 18
1.2. Background 22
1.2.1. Formulation 22
1.2.2. Problems 24
1.3. Contributions 26
1.4. Outline 27
Chapter Ⅱ. Preliminaries 28
2.1. Evaluation Metrics 28
2.1.1. Peak-Signal-to-Noise Ratio 29
2.1.2. Structural Similarity Index Measure 30
2.2. Video Super Resolution 31
2.2.1. VSRNet 32
2.2.2. VESPCN 32
2.2.3. FRVSR 33
2.2.4. TOFlow 34
2.2.5. DUF 34
2.2.6. TDAN 35
2.2.7. EDVR 36
2.2.8. RBPN 36
2.2.9. RLSP 37
2.2.10. RRN 38
2.2.11. SOFVSR 38
2.2.12. IconVSR 39
2.2.13. BasicVSR++ 40
2.2.14. OVSR 41
2.2.15. RTA 42
2.2.16. VRT 42
2.2.17. RVRT 43
2.3. BasicVSR 44
2.4. Attention Mechanism 47
Chapter Ⅲ. Proposed Method 49
3.1. Network Architecture 50
3.1.1. Data Preprocessing 50
3.1.2. Network Architecture 52
3.2. The proposed Refinement Module 56
3.2.1. Feature refinement/alignment 57
3.2.2. Extra feature fusion 60
Chapter Ⅳ. Experimental Results 63
4.1. Datasets 63
4.2. Training Details 64
4.3. Comparisons with State-of-the-Art 66
4.4. Ablation Studies 70
4.4.1. Image refinement 70
4.4.2. Attention and feature fusion 83
4.4.3. BasicVSR++ with refinement module 84
4.5. Discussions 86
Chapter Ⅴ. Conclusion and Future Works 88
5.1. Conclusion 88
5.2. Future Works 90
References 91
ABSTRACT IN KOREAN 104
Curriculum Vitae 106
Table 4.1. Experiment environment for training models. 65
Table 4.2. Quantitative comparison (average PSNR/SSIM) 67
Table 4.3. Comparison of models that perform feature extraction before alignment. 76
Table 4.4. Evaluations of Refinement module components. 84
Table 4.5. The performance comparison on BasicVSR++. 85
Fig. 1.1. Various applications of super-resolution. 19
Fig. 1.2. The pipeline of the deep learning-based VSR method. 24
Fig. 2.1. BasicVSR architecture. 44
Fig. 3.1. Example of flipping sequence: 5 frames to 10 frames. 51
Fig. 3.2. Data preprocessing for training. 51
Fig. 3.3. Overall architecture of the proposed scheme. 53
Fig. 3.4. The proposed refinement module. 58
Fig. 3.5. Types of feature fusion. 61
Fig. 3.6. Feature warping without and with extra feature fusion. 61
Fig. 4.6. PSNR comparison of feature extractor structure. 77
Fig. 4.7. Feature extractor comparison. 78
Fig. 4.8. Featuremap flow of change. 79
Fig. 4.9. Optical flow comparison. 81
Fig. 4.10. Tradeoff of RCAB. 82
Fig. 4.11. Effect of attention (att) and feature fusion (ff). 84