超解析度成像法的目的是提升影像的空間解析度，恢復影像次像素中遺失的細節。傳統提升影像解析度的方法多使用線性或非線性內插，運行快速、耗費運算資源低，但在影像成果較模糊，高頻細節的復原並不理想。隨著類神經網路的發展，針對影像處理的卷積神經網路大量運用於超解析度成像法。許多卷積神經網路的超解析度成像法為監督式學習方法，需要大量成對的外部影像集將模型訓練至可提升特定特徵目標如人臉、風景或建物的影像解析度，因此訓練過程運算需求高且耗時長，訓練成果與外部影像集的豐富程度和訓練迭代次數相關。相對而言，非監督超解析度成像法不需要耗費高成本獲得成對的外部影像集來學習影像從低解析度到高解析度的映射，大幅減少計算資源，使非監督學習對未知的新數據有更好的適應性，提高實際應用場景中的可用性和應用性。
本論文針對三種非監督學習方法Zero-shot Super-resolution (ZSSR)、Deep Image Prior (DIP) 及Degradation-aware Super-resolution (DASR)，比較其提升Sentinel-2及SPOT衛星影像之空間解析度的效能。實驗結果顯示，非監督學習方法相較於雙三次內插法在峰值訊噪比(Peak Signal-to-Noise Ratio, PSNR)、結構相似性(Structural Similarity, SSIM)及特徵相似性(Feature Similarity, FSIM)有所提升，與耗費訓練資源的監督式學習方法相比仍展現優勢。成果也顯示三種非監督學習方法中，Degradation-aware Super-resolution在光學衛星影像中表現最佳。

The purpose of super-resolution imaging is to improve the spatial resolution of images, restore lost details in the sub-pixel levels. The traditional methods based on linear or nonlinear interpolations are fast and consume fewer computing resources. But the results are usually blur and not suitable for high-frequency details recovery. With the development of artificial intelligent, the convolutional neural networks (CNN) are widely adopted for image super-resolution. Most of CNN-based super-resolution algorithms are supervised learning. They need huge amount of external training image pairs to train the model and improve the image resolution for specific feature such as faces, scenery or buildings. Therefore, they all require high computational power and long training time. The richness of the external image set and the number of training iterations are related to the effectiveness. On the other hand, unsupervised super-resolution algorithms aim to learn the relationship from low-resolution to high-resolution images only from the test image itself. Unsupervised methods do not need to collect paired training data, and can reduce computing resources significantly. This also makes a flexible adaption on new and unseen data, increased accessibility and applicability in real-world scenarios.
In this study, three different unsupervised super-resolution methods, Zero-shot Super-resolution (ZSSR), Deep Image Prior (DIP) and Degradation-aware Super-resolution (DASR), are implemented and applied to improve the spatial resolution of Sentinel-2 and SPOT images. The experimental results show that unsupervised methods can improve Peak Signal-to-noise Ratio (PSNR), Structural Similarity (SSIM) and Feature Similarity (FSIM) compared with bicubic interpolation, also competitively to supervised methods that consume expansive training resources. The results also show that Degradation-aware Super-resolution performs the best among these three unsupervised methods.

[1] Z. Wang, J. Chen, and S. C. H. Hoi, "Deep Learning for Image Super-Resolution: A Survey," Ieee T Pattern Anal, vol. 43, no. 10, pp. 3365-3387, 2021.
[2] H. Greenspan, "Super-Resolution in Medical Imaging," (in English), Comput J, vol. 52, no. 1, pp. 43-63, 2009.
[3] J. S. Isaac and R. Kulkarni, "Super resolution techniques for medical image processing," in 2015 International Conference on Technologies for Sustainable Development (ICTSD): IEEE, pp. 1-6.
[4] L. Zhang, H. Zhang, H. Shen, and P. Li, "A super-resolution reconstruction algorithm for surveillance images," Signal Processing, vol. 90, no. 3, pp. 848-859, 2010.
[5] K. Nasrollahi and T. B. Moeslund, "Super-resolution: a comprehensive survey," Machine Vision and Applications, vol. 25, no. 6, pp. 1423-1468, 2014.
[6] S. Vitale and G. Scarpa, "A Detail-Preserving Cross-Scale Learning Strategy for CNN-Based Pansharpening," Remote Sensing, vol. 12, no. 3, p. 348, 2020.
[7] M. Gargiulo, A. Mazza, R. Gaetano, G. Ruello, and G. Scarpa, "Fast Super-Resolution of 20 m Sentinel-2 Bands Using Convolutional Neural Networks," Remote Sensing, vol. 11, p. 2635, 2019.
[8] W. Ma, Z. Pan, F. Yuan, and B. Lei, "Super-Resolution of Remote Sensing Images via a Dense Residual Generative Adversarial Network," Remote. Sens., vol. 11, p. 2578, 2019.
[9] J. Gu, X. Sun, Y. Zhang, K. Fu, and L. Wang, "Deep Residual Squeeze and Excitation Network for Remote Sensing Image Super-Resolution," Remote Sensing, vol. 11, p. 1817, 2019.
[10] A. L. Maas, "Rectifier Nonlinearities Improve Neural Network Acoustic Models," 2013.
[11] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," (in English), P Ieee, vol. 86, no. 11, pp. 2278-2324, 1998.
[12] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, no. 6088, pp. 533-536, 1986.
[13] C. Dong, C. C. Loy, K. M. He, and X. O. Tang, "Image Super-Resolution Using Deep Convolutional Networks," (in English), Ieee T Pattern Anal, vol. 38, no. 2, pp. 295-307, 2016.
[14] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778.
[15] C. Ledig et al., "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105-114.
[16] B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution," in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1132-1140.
[17] A. Shocher, N. Cohen, and M. Irani, "Zero-Shot Super-Resolution Using Deep Internal Learning," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3118-3126.
[18] D. Ulyanov, A. Vedaldi, and V. Lempitsky, "Deep Image Prior," (in English), Int J Comput Vision, vol. 128, no. 7, pp. 1867-1888, 2020.
[19] Y. Yuan, S. Liu, J. Zhang, Y. Zhang, C. Dong, and L. Lin, "Unsupervised Image Super-Resolution Using Cycle-in-Cycle Generative Adversarial Networks," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 814-81409.
[20] Y. B. Zhang, S. Y. Liu, C. Dong, X. F. Zhang, and Y. Yuan, "Multiple Cycle-in-Cycle Generative Adversarial Networks for Unsupervised Image Super-Resolution," (in English), Ieee T Image Process, vol. 29, pp. 1101-1112, 2020.
[21] L. Wang et al., "Unsupervised Degradation Representation Learning for Blind Super-Resolution," 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10576-10585, 2021.
[22] L. Liebel and M. Körner, "SINGLE-IMAGE SUPER RESOLUTION FOR MULTISPECTRAL REMOTE SENSING DATA USING CONVOLUTIONAL NEURAL NETWORKS," Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., vol. XLI-B3, pp. 883-890, 2016.
[23] H. Song, Q. Liu, G. Wang, R. Hang, and B. Huang, "Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 3, pp. 821-829, 2018.
[24] J. Shermeyer and A. V. Etten, "The Effects of Super-Resolution on Object Detection Performance in Satellite Imagery," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1432-1441.
[25] J. Kim, J. K. Lee, and K. M. Lee, "Accurate Image Super-Resolution Using Very Deep Convolutional Networks," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1646-1654.
[26] Y. Luo, L. Zhou, S. Wang, and Z. Wang, "Video Satellite Imagery Super Resolution via Convolutional Neural Networks," IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 12, pp. 2398-2402, 2017.
[27] F. Dorr, "Satellite Image Multi-Frame Super Resolution Using 3D Wide-Activation Neural Networks," Remote Sensing, vol. 12, p. 3812, 2020.
[28] J. Yu et al., "Wide activation for efficient and accurate image super-resolution," arXiv preprint arXiv:1808.08718, 2018.
[29] M. Märtens, D. Izzo, A. Krzic, and D. Cox, "Super-resolution of PROBA-V images using convolutional neural networks," Astrodynamics, vol. 3, no. 4, pp. 387-402, 2019.
[30] J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks," in 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242-2251.
[31] P. Wang, H. Zhang, F. Zhou, and Z. Jiang, "Unsupervised Remote Sensing Image Super-Resolution Using Cycle CNN," in IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, pp. 3117-3120.
[32] M. Qin et al., "Achieving Higher Resolution Lake Area from Remote Sensing Images Through an Unsupervised Deep Learning Super-Resolution Method," Remote Sensing, vol. 12, no. 12, p. 1937, 2020.
[33] M. Zontak and M. Irani, "Internal statistics of a single natural image," in CVPR 2011, pp. 977-984.
[34] D. P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," doi: 10.48550/arXiv.1412.6980.
[35] J. Gu, H. Lu, W. Zuo, and C. Dong, "Blind Super-Resolution With Iterative Kernel Correction," presented at the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[36] A. v. d. Oord, Y. Li, and O. Vinyals, "Representation learning with contrastive predictive coding," doi: 10.48550/arXiv:1807.03748.
[37] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, "Momentum Contrast for Unsupervised Visual Representation Learning," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9726-9735.
[38] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 416-423 vol.2.
[39] L. van der Maaten and G. Hinton, "Viualizing data using t-SNE," Journal of Machine Learning Research, vol. 9, pp. 2579-2605, 2008.
[40] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," (in English), Ieee T Image Process, vol. 13, no. 4, pp. 600-612, 2004.
[41] L. Zhang, L. Zhang, X. Q. Mou, and D. Zhang, "FSIM: A Feature Similarity Index for Image Quality Assessment," (in English), Ieee T Image Process, vol. 20, no. 8, pp. 2378-2386, 2011.
[42] Y. Kang, L. Pan, M. Sun, X. Liu, and Q. Chen, "Destriping high-resolution satellite imagery by improved moment matching," International Journal of Remote Sensing, vol. 38, pp. 6346-6365, 2017.
[43] "Sentinel-2 User Handbook," European Space Agency (ESA), 2015.
[44] "SPOT 6/7 Imagery - User Guide," Airbus Defence and Space Intelligence, France, 2013.
[45] E. Agustsson and R. Timofte, "NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study," in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1122-1131.
[46] R. Timofte et al., "NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results," in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1110-1121.
[47] W. Shi et al., Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. 2016.
[48] L. C. Pickup, Machine learning in multi-frame image super-resolution. 2007.
[49] M. R. Arefin et al., "Multi-Image Super-Resolution for Remote Sensing using Deep Recurrent Networks," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW): IEEE Computer Society, pp. 816-825.