跳到主要內容

簡易檢索 / 詳目顯示

回結果列表
研究生： 張郁欣
Yu-Hsin Chang
論文名稱： 基於卷積神經網路之超解析方法對高光譜遙測影像分類的影響
Effect of CNN-based Super-Resolution on Hyperspectral Image Classification
指導教授： 任玄
Hsuan Ren
口試委員:
學位類別： 碩士
Master
系所名稱： 太空及遙測研究中心 - 遙測科技碩士學位學程
Master of Science Program in Remote Sensing Science and Technology
論文出版年： 2024
畢業學年度： 112
語文別： 英文
論文頁數： 76
中文關鍵詞： 影像超解析 、衛星影像 、高光譜影像分類 、卷積神經網路
外文關鍵詞： super-resolution, satellite image, hyperspectral image classification, convolutional neural network
相關次數： 點閱：13  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 在地球環境觀測的領域，遙測影像是一個能提供大尺度之時間與空間觀測資料的利器。其中，高光譜影像的波段數為數十個至數百個波段，能夠提供目標地物的細微光譜特徵，並以此做為地物分類的依據。然而，在同樣能量強度的入射光源下，每個波段能分配到的能量比較少，使得高光譜影像的空間解析度被犧牲。超解析影像處理是一種電腦視覺領域的影像處理技術，目的在於提升影像的空間解析度，目前已被廣泛應用在各式各樣的領域，例如文字辨識、安全監控、手機的修圖軟體等等。近年來超解析影像模型的運算能力越來越強大，將這樣的技術應用於提升高光譜遙測影像的空間解析度，或許可以協助改善地物分類的準確度。
    本論文聚焦在以卷積神經網路為架構的超解析影像模型，例如MS-LapSRN與EDSR，並比較這些超解析模型對於提升高光譜遙測影像之空間解析度的效果。本文使用Sentinel-2與SPOT-6/7光學衛星影像作為訓練及測試超解析模型的資料集，結果顯示MS-LapSRN_D5R8的模型在峰值訊噪比 (PSNR) 與結構相似性 (SSIM) 指標上有較好的表現。接著，再以高光譜遙測影像分類基準資料集，例如Indian Pines 資料集與Pavia University資料集，以最小歐幾里得距離法以及傳統類神經網路進行監督式純像元分類，並比較提升空間解析度之前與之後，在地物分類任務的表現。本研究針對兩組高光譜影像之實驗成果顯示，影像超解析不一定能提升高光譜影像分類的準確率，針對其影響的規律需要更進一步的研究來探討。

    In the field of earth environment observation, remote sensing imagery is a powerful tool that can provide large-scale temporal and spatial observation data. Among them, hyperspectral images have hundreds of bands, which can provide subtle spectral characteristics of target objects and serve as the basis for land cover classification. However, under the same energy intensity of the incident light source, the amount of energy for each band is reduced, resulting in the sacrifice of spatial resolution of hyperspectral images. Super-resolution (SR) image processing is an image processing technology in the field of computer vision that aims to improve the spatial resolution of images. It has been widely used in a variety of fields, such as text recognition, security monitoring, and mobile phone photo editing software. In recent years, the computing power of super-resolution image models has been significantly improved. Applying such technology to improve the spatial resolution of hyperspectral remote sensing images might help improve the accuracy of land cover classification.
    This study focuses on image super-resolution models that are based on convolutional neural networks, such as MS-LapSRN and EDSR, and compares the effects of these super-resolution models on improving the spatial resolution of hyperspectral remote sensing images. The optical satellite imagery, Sentinel-2 and SPOT-6/7, were used as training and testing dataset for SR model. The results show that MS-LapSRN_D5R8 model has better performance in peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). Furthermore, hyperspectral image classification benchmark datasets, Indian Pines dataset and Pavia University dataset, were used to compare the performance in land cover classification tasks before and after improving the spatial resolution. Minimum Euclidean distance (MED) and traditional neural network (NN) was used as supervised pure-pixel classification method. It was observed in the context of this study that image super-resolution does not necessarily improve the accuracy of hyperspectral classification, and further studies are needed to investigate this issue.

    摘 要 i Abstract ii Contents iii List of Figures v List of Tables viii Chapter 1 Introduction 1 1.1 Overview 1 1.2 Motivation 1 Chapter 2 Literature Review 2 2.1 Image Spatial Resolution Enhancement 2 2.2 Image Fusion 2 2.2.1 Image Super-Resolution 2 2.3 CNN based Image Super-Resolution 3 2.3.1 Pre-upscaling SR 4 2.3.2 Post-upscaling SR 4 2.3.3 Progressive upscaling SR 5 2.3.4 Iterative up-and-down scaling SR 5 2.4 Hyperspectral imagery (HSI) 6 Chapter 3 Research Materials 8 3.1 Image Super-Resolution Dataset 8 3.2 Image Classification Dataset 10 Chapter 4 Methodology 14 4.1 Image Super-Resolution Algorithm 14 4.1.1 Multi-Scale Laplacian Pyramid Super-Resolution Network 14 4.1.2 Enhanced Deep Super-Resolution Network 20 4.2 Image Classification Method 21 4.2.1 Minimum Euclidean Distance 21 4.2.2 Neural Network 22 Chapter 5 Result and Discussion 24 5.1 Image Super-Resolution Quality Assessment 24 5.2 Image Super-Resolution Result 25 5.3 Image Classification Assessment 37 5.4 Image Classification Result 38 5.4.1 Figure Interpretation 38 5.4.2 Classification Result Comparison 39 Chapter 6 Conclusions 59 6.1 Conclusions 59 6.2 Future Works 59 References 61

    [1] L. S. Romero, J. Marcello, and V. Vilaplana, "Super-Resolution of Sentinel-2 Imagery Using Generative Adversarial Networks," Remote Sens., Article vol. 12, no. 15, p. 25, Aug 2020, Art no. 2424, doi: 10.3390/rs12152424.
    [2] Z. H. Wang, J. Chen, and S. C. H. Hoi, "Deep Learning for Image Super-Resolution: A Survey," IEEE Trans. Pattern Anal. Mach. Intell., Article vol. 43, no. 10, pp. 3365-3387, Oct 2021, doi: 10.1109/tpami.2020.2982166.
    [3] C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part IV 13, 2014: Springer, pp. 184-199.
    [4] C. Dong, C. C. Loy, K. He, and X. Tang, "Image Super-Resolution Using Deep Convolutional Networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp. 295-307, 2016, doi: 10.1109/TPAMI.2015.2439281.
    [5] Y. Tai, J. Yang, X. Liu, and C. Xu, "MemNet: A Persistent Memory Network for Image Restoration," in 2017 IEEE International Conference on Computer Vision (ICCV), 22-29 Oct. 2017 2017, pp. 4549-4557, doi: 10.1109/ICCV.2017.486.
    [6] Y. Tai, J. Yang, and X. Liu, "Image Super-Resolution via Deep Recursive Residual Network," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July 2017 2017, pp. 2790-2798, doi: 10.1109/CVPR.2017.298.
    [7] J. Kim, J. K. Lee, and K. M. Lee, "Deeply-Recursive Convolutional Network for Image Super-Resolution," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June 2016 2016, pp. 1637-1645, doi: 10.1109/CVPR.2016.181.
    [8] C. Dong, C. C. Loy, and X. Tang, "Accelerating the super-resolution convolutional neural network," in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, 2016: Springer, pp. 391-407.
    [9] W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June 2016 2016, pp. 1874-1883, doi: 10.1109/CVPR.2016.207.
    [10] B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution," The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017.
    [11] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July 2017 2017, pp. 105-114, doi: 10.1109/CVPR.2017.19.
    [12] W. Han, S. Chang, D. Liu, M. Yu, M. Witbrock, and T. S. Huang, "Image Super-Resolution via Dual-State Recurrent Networks," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18-23 June 2018 2018, pp. 1654-1663, doi: 10.1109/CVPR.2018.00178.
    [13] T. Tong, G. Li, X. Liu, and Q. Gao, "Image Super-Resolution Using Dense Skip Connections," in 2017 IEEE International Conference on Computer Vision (ICCV), 22-29 Oct. 2017 2017, pp. 4809-4817, doi: 10.1109/ICCV.2017.514.
    [14] W. S. Lai, J. B. Huang, N. Ahuja, and M. H. Yang, "Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks," IEEE Trans. Pattern Anal. Mach. Intell., Article vol. 41, no. 11, pp. 2599-2613, Nov 2019, doi: 10.1109/tpami.2018.2865304.
    [15] Y. Wang, F. Perazzi, B. McWilliams, A. Sorkine-Hornung, O. Sorkine-Hornung, and C. Schroers, "A Fully Progressive Approach to Single-Image Super-Resolution," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 18-22 June 2018 2018, pp. 977-97709, doi: 10.1109/CVPRW.2018.00131.
    [16] M. Irani and S. Peleg, "Improving resolution by image registration," CVGIP: Graphical models and image processing, vol. 53, no. 3, pp. 231-239, 1991.
    [17] M. Haris, G. Shakhnarovich, and N. Ukita, "Deep Back-Projection Networks for Super-Resolution," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18-23 June 2018 2018, pp. 1664-1673, doi: 10.1109/CVPR.2018.00179.
    [18] Z. Li, J. Yang, Z. Liu, X. Yang, G. Jeon, and W. Wu, "Feedback Network for Image Super-Resolution," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019 2019, pp. 3862-3871, doi: 10.1109/CVPR.2019.00399.
    [19] M. Haris, G. Shakhnarovich, and N. Ukita, "Recurrent Back-Projection Network for Video Super-Resolution," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019 2019, pp. 3892-3901, doi: 10.1109/CVPR.2019.00402.
    [20] USGS EROS Archive - Earth Observing One (EO-1) - Hyperion [Online] Available: https://www.usgs.gov/centers/eros/science/usgs-eros-archive-earth-observing-one-eo-1-hyperion
    [21] AVIRIS (Airborne Visible/Infrared Imaging Spectrometer) [Online] Available: https://aviris.jpl.nasa.gov/aviris
    [22] Indian Pines Hyperspectral Image Dataset [Online] Available: https://engineering.purdue.edu/~biehl/MultiSpec/hyperspectral.html
    [23] Hyperspectral Remote Sensing Scenes [Online] Available: https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes
    [24] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," In CVPR, 2016.

    QR CODE
    :::