在本論文中，我們探討H.264的編碼核心技術，H.264編碼器可以對每一個Marcoblock選擇不同的編碼模式，本論文中，我們將探討如何利用Rate-Distortion Optimization的觀念。論文中會介紹如何用Rate-Distortion Optimization來選擇最佳編碼模式。由於多重編碼模式和其他工具的引進，使得H.264比以往視訊標準有更好的編碼效率；但相對的也要付出更多的計算資源，以獲得更好的視訊品質。
由於各種編碼模式出現的機率不同，我們主要是根據Marcoblock mode 出現頻率的不同而提出模式選擇策略，並且提供一系列的簡化準則：如Dynamic mode checking、 early termination、 pseudo motion vector等；根據這些準則，期望能在付出一點畫質損失的代價而增加編碼效率。

In this thesis, we study the core of the encoding process for H.264. To improve the performance of H.264 encoding, we allow H.264 encoder to select different decision modes for variant encoding process in each marcoblock. Based on the multiple encoding modes and more developed tools, H.264 has better coding efficiency than the previous video standard; however, such an improvement on H.264 needs more computation resources to get better video quality.
In this thesis, we propose the mode selection strategy, early termination criterion, fast intra/inter mode switch, dynamic mode checking, and pseudo motion vector prediction to speed up the encoding. The mode selection strategy is the core of this study. The selection criterion is based on Rate-Distortion Optimization, and the strategy is based on the principle of probability distribution.
In experiments, we make a comparison between the standard method and proposed method and achieve the balance between the image quality and the encoding speed.

[1] Chan, M. H., Y. B. Yu, and A. G. Constantinides, “Variable size block matching motion compensation with applications to video coding”, Communications, Speech and Vision, IEE Proceedings I, vol.137, pp.205-212, Aug. 1990.
[2] Sullivan, G. J. and T. Wiegand, “Rate-distortion optimization for video compression,” IEEE Signal Processing Magazine, vol.15, pp.74-90, Nov. 1998.
[3] Injong, R., G. R. Martin, S. Muthukrishnan, and R. A. Packwood, “Quadtree-structured variable-size block-matching motion estimation with minimal error,” IEEE Trans. on circuits and systems for video technology, vol.10, pp.42-50, Feb. 2000.
[4] ITU Telecom, Video Coding for Low Bitrate Communication, ITU-T Recommendation H.263, 1996.
[5] ITU-T Video Coding Experts Group, H.26L Test Model Long Term Number 9, TML-9, Dec. 2001.
[6] ITU-T Recommendation H.264/ISO/IEC 11496-10, Advanced Video Coding, Final Committee Draft, Document JVT-E022, Sep. 2002.
[7] MPEG Video Group, MPEG-4 Video Verification Model, Version 18.0, ISO/IEC JTC1/SC29/WG11 N3908, Jan. 2001.
[8] Seferidis, V. and M. Ghanbari, “Generalised block-matching motion estimation using quad-tree structured spatial decomposition,” IEE Proc. Vision, Image, and Signal Processing, vol.141, pp.446-452, Dec. 1994.
[9] Packwood, R. A., M. K. Steliaros, and G. R. Martin, “Variable size block matching motion compensation for object-based video coding,“ in Proc. on 6th Int. Conf. Image Processing and Its Applications, July 1997, vol.1, pp.56-60.
[10] Tu, Y. K., J. F. Yang, Y. N. Shen, and M. T. Sun, “Fast variable-size block motion estimation using merging procedure with an adaptive threshold,“ in Proc. IEEE Int. Conf. Multimedia and Expo., Baltimore, vol.2, July 2003, pp.789-792.
[11] Puri, A., H.-M. Hang, and D. L. Schilling, “Interframe coding with variable block-size motion compensation,'' in Proc. IEEE Global Commun. Conf., Tokyo, Japan, Nov.1987, pp.65-69.
[12] Lee, J. and B. Jeon, “Fast mode decision for H.264,“ in Proc. 2004 IEEE Int. Conf. Multimedia and Expo, ICME ''04, Jun. 2004, vol.2, pp.1131-1134.
[13] Han, K.-H. and Y. L. Lee, “Fast macroblock mode decision in H.264 “ in Proc. TENCON 2004, Nov.2004, vol.1, pp.347-350.
[14] Kim, Y.-H., J.-W. Yoo, S.-W. Lee, J. Shin, J. Park, and H.-K. Jung, “Adaptive mode decision for H.264 encoder“, Electronics Letters, vol.40, no.19, pp.1172-1173, Sep. 2004.
[15] Richardson, I. E. G., H.264 and MPEG-4 Video Compression, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, 2003.