Title Page
Contents
ABSTRACT 8
Ⅰ. Introduction 9
Ⅱ. Previous Research 13
2.1. Image and point cloud semantic segmentation datasets 13
2.2. Camera-LiDAR fusion based semantic segmentation networks 16
Ⅲ. Construction of Multi-segmentation Dataset 22
3.1. Sensors and data collection 22
3.2. Data annotation 24
3.3. Data analysis 25
Ⅳ. Camera-LiDAR Fusion based Multi-segmentation Network 27
4.1. Framework overview 27
4.2. Modal-specific architectures 28
4.2.1. Camera image branch 28
4.2.2. LiDAR point and voxel branches 29
4.3. Image-point-voxel feature fusion module 32
Ⅴ. Experiments 35
5.1. Experimental environment 35
5.1.1. Dataset 35
5.1.2. Implementation details 35
5.1.3. Evaluation metrics 36
5.2. Experimental results 37
Ⅵ. Conclusion 45
References 46
Abstract (in Korean) 50
〈Table 5-1〉 Comparison on multi-segmentation test set 38
〈Table 5-2〉 Performance comparison by distance 39
〈Figure 1-1〉 Image Semantic Segmentation 9
〈Figure 1-2〉 Pointcloud Semantic Segmentation 10
〈Figure 2-1〉 Image semantic segmentation datasets for autonomous driving 14
〈Figure 2-2〉 Pointcloud semantic segmentation datasets for autonomous driving 15
〈Figure 2-3〉 Camera data representation method 16
〈Figure 2-4〉 LiDAR data representation method 17
〈Figure 2-5〉 Architecture of VIASeg 18
〈Figure 2-6〉 Architecture of FuseSeg 19
〈Figure 2-7〉 Architecture of PMF 20
〈Figure 2-8〉 Architecture of LIFSeg 21
〈Figure 3-1〉 Data collection platform 22
〈Figure 3-2〉 The area of data collection 23
〈Figure 3-3〉 Diversity in data acquisition time and location 23
〈Figure 3-4〉 Result of time synchronization 24
〈Figure 3-5〉 Pixel and point statistics by class 26
〈Figure 4-1〉 Overall network architecture 28
〈Figure 4-2〉 Residual learning 29
〈Figure 4-3〉 Multi Layer Perceptron 30
〈Figure 4-4〉 Kernel point convolution 31
〈Figure 4-5〉 Sparse convolution 31
〈Figure 4-6〉 Geometric relationship between the camera and LiDAR 33
〈Figure 4-7〉 Image-point-voxel feature fusion module 34
〈Figure 5-1〉 Qualitative results of image semantic segmentation 40
〈Figure 5-2〉 Qualitative results of point cloud semantic segmentation (1) 42
〈Figure 5-3〉 Qualitative results of point cloud semantic segmentation (2) 43
〈Figure 5-4〉 Qualitative results of point cloud semantic segmentation (3) 44