為了將影像完整呈現於各種尺寸的輸出裝置，且盡量減少視覺上的
扭曲變形，許多基於內容之影像尺寸調整機制被提出，如何有效地評估
各種方法的效果成為一項重要任務。本研究提出一個基於物件重要程度
的影像尺寸調整評估機制，透過語義分割方法將影像中的所有像素點分
類，根據語義中的類別，給予該所在區域不同的視覺重要程度，依此做
為人眼視覺對於該區域受破壞的敏感度衡量，希冀獲致更貼近使用者主
觀感受的顯著圖，並將其應用於長寬比相似性畫質衡量演算法以提升準
確度。我們另外觀察到人眼觀看無前景物影像時容易受到畫面整體資訊
損失的影響，因此提出無明顯前景物資訊損失懲罰調整策略。我們先利
用語義資訊判斷場景中有無明顯前景物，再給予不同大小級別的資訊損
失懲罰，提高無明顯前景物場景的評分準確度。實驗結果顯示，本研究
能有效評估影像尺寸調整機制，與現有方法相較有更高的準確度。

Many image retargeting methods have been proposed to resize images to
fit in various sizes of display devices with less perceptual distortion. Assessing
the quality of retargeted images has thus become an important task for
developing such methods. In this research, we propose an image retargeting
quality assessment (IRQA) based on importance of objects. We utilize
semantic segmentation to classify pixels, which are assigned with different
importance values representing the sensitivity of human eyes to distortion. A
visual saliency map is created to better fit the subjective perception of humans
and is then used in the evaluation method called “Aspect Ratio Similarity”
(ARS) to improve its accuracy. Furthermore, as observing that human eyes
tend to be affected more by the global information loss in images in which
there is no obvious foreground object, we propose the strategy of information
loss adjustment in such images. We first utilize semantic information to
determine whether a foreground object exists and then adopt different degrees
of information loss penalty to improve the accuracy of the assessment. The
experimental results show that the proposed approach is effective in
evaluating the image retargeting methods and outperforms existing quality
assessment methods.

[1] M. Rubinstein, S. Ariel, and A. Shai. "Improved seam carving for video
retargeting." ACM transactions on graphics, vol. 27, no. 3, 2008.
[2] P. Krähenbühl, M. Lang, A. Hornung, and M. H. Gross, “A system for
retargeting of streaming video,” ACM Transactions on Graphics, vol. 28, no. 5,
2009.
[3] Y. Pritch, K. Eitam, and P. Shmuel. "Shift-map image editing." 2009 IEEE
12th international conference on computer vision. IEEE, 2009.
[4] Y.-S. Wang, C.-L. Tai, O. Sorkine, and T.-Y. Lee, “Optimized scale-andstretch for image resizing,” ACM transactions on graphics, vol. 27, no. 5, pp.
118, 2008.
[5] L. Wolf, G. Moshe, and C. Daniel. "Non-homogeneous content-driven
video-retargeting." 2007 IEEE 11th international conference on computer
vision. IEEE, 2007.
[6] Y. Zhang, W. Lin, X. Zhang, Y. Fang and L. Li, "Aspect ratio similarity
(ARS) for image retargeting quality assessment." 2016 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE,
2016.
[7] Y. Zhang, Y. Fang, W. Lin, X. Zhang, and L. Li, “Backward registrationbased aspect ratio similarity for image retargeting quality assessment,” IEEE
Transactions on image processing, vol. 25, no. 9, pp. 4286–4297, 2016.
[8] Y. Zhang, W. Lin, Q. Li, W. Cheng, and X. Zhang, “Multiple-level featurebased measure for retargeted image quality,” IEEE Transactions on image
processing, vol. 27, no. 1, pp. 451–463, Jan. 2018.
[9] M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative
study of image retargeting,” ACM transactions on graphics, vol. 29, no. 5, pp.
160:1–160:10, 2010.
[10] M. Rubinstein, A. Shamir, and S. Avidan, “Multi-operator media
retargeting,” ACM transactions on graphics, vol. 28, no. 3, 2009.
[11] D.-Y. Wei, “Content-Based Multi-Operator Retargeting and Its Quality
Evaluation,” NCU, 2019. [Online]. Available:
http://ir.lib.ncu.edu.tw/handle/987654321/81272
[12] D. Simakov, Y. Caspi, E. Shechtman, and M. Irani, "Summarizing visual
data using bidirectional similarity." 2008 IEEE Conference on Computer
Vision and Pattern Recognition. IEEE, 2008.
[13] B. S. Manjunath, J.-R. Ohm, V. V. Vasudevan, and A. Yamada, “Color and
texture descriptors,” IEEE Transactions on circuits and systems for video
technology, vol. 11, no. 6, pp. 703–715, Jun. 2001.
[14] E. Kasutani and A. Yamada, "The MPEG-7 color layout descriptor: a
compact image feature description for high-speed image/video segment
retrieval." Proceedings 2001 International Conference on Image Processing
(Cat. No. 01CH37205). vol. 1. IEEE, 2001.
[15] C. Liu, J. Yuen, and A. Torralba, “SIFT flow: Dense correspondence across
scenes and its applications,” IEEE transactions on pattern analysis and machine
intelligence, vol. 33, no. 5, pp. 978–994, May 2011.
[16] O. Pele and M. Werman, “Fast and robust earth mover’s distances,” 2009
IEEE 12th international conference on computer vision, IEEE, 2009.
[17] C.-C. Hsu, C.-W. Lin, Y. Fang, and W. Lin, “Objective quality assessment
for image retargeting based on perceptual geometric distortion and information
loss,” IEEE Journal of Selected Topics in Signal Processing, vol. 8, no. 3, pp.
377–389, Jun. 2014.
[18] Z. Peng, J. Qiuping, F. Shao, W. Gao, W. Lin, "Lggd+: Image retargeting
quality assessment by measuring local and global geometric distortions." IEEE
Transactions on Circuits and Systems for Video Technology, 2021.
[19] Y. Fang, Z. Chen, W. Lin, and C.-W. Lin, “Saliency detection in the
compressed domain for adaptive image retargeting,” IEEE Transactions on
Image Processing, vol. 21, no. 9, pp. 3888–3901, Sep. 2012.
[20] S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu, J. Feng, T. Xiang,
P. Torr, L. Zhang, “Rethinking Semantic Segmentation from a Sequence-toSequence Perspective with Transformers,” Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition (CVPR), June. 2021.
[21] B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, A. Torralba, "Semantic
understanding of scenes through the ade20k dataset." International Journal of
Computer Vision, vol. 127, no. 3, pp. 302-321, 2019.
[22] Mmsegmentation Contributors, MMSegmentation: OpenMMLab
Semantic Segmentation Toolbox and Benchmark. https://github.com/openmmlab/mmsegmentation, 2020. [Online]. Available: https://github.com/openmmlab/mmsegmentation
[23] M. G. Kendall, “A new measure of rank correlation,” Biometrika, vol. 30,
pp. 81–93, Jun. 1938.
[24] L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention
for rapid scene analysis,” IEEE Transactions on pattern analysis and machine
intelligence, vol. 20, no. 11, pp. 1254–1259, Nov. 1998.
[25] D. DeCarlo and A. Santella, “Stylization and abstraction of photographs,”
ACM transactions on graphics, vol. 21, no. 3, pp. 769–776, 2002.
[26] R. Achanta and S. Süsstrunk, "Saliency detection for content-aware image
resizing." 2009 16th IEEE international conference on image processing (ICIP).
IEEE, 2009.
[27] Y. Niu, J. Chen, and W. Guo, “Meta-metric for saliency detection
evaluation metrics based on application preference,” Multimedia Tools and
Applications, vol. 77, no. 20, pp. 26351–26369, 2018