近年容器化（containerized）、微服務架構（microservices architecture）興起，軟體架構的複雜度日益增加，通常會使用容器管理工具自動部署及管理容器，開源的Kubernetes是目前最廣泛被使用的容器管理工具。以往叢集內服務間的同步溝通以HTTP/1.1（Hypertext Transfer Protocol Version 1.1）為主，隨著HTTP/2（Hypertext Transfer Protocol Version 2）的發布，許多系統改用HTTP/2做為服務間溝通的方式，期望透過持久性連線（persistent connection）提升傳輸效率，然而Kubernetes內部負載平衡器kube-proxy屬於network load balancer，只能處理L4的通訊協定，HTTP/2是在L7實作持久性連線的功能，透過此通訊協定連上Pod後，在後續的資料傳輸過程中沒辦法將流量導向到其他Pod，也失去自動擴展功能的意義。為解決此問題，許多文章建議使用服務網格（Service Mesh），透過其L7負載平衡器平衡流量，本篇論文設計數個實驗，比較Kubernetes使用kube-proxy與使用服務網格的進行負載平衡的效能差異，在Kubernetes使用服務網格平衡HTTP/2流量，須在合適的場景下才能提升系統效益。

In recent years, with the rise of containerization and microservice architecture, software architecture is getting more complex. Container orchestration systems are used to automatically deploy and manage multiple containers. Kubernetes is an open-source project and the most widely used in the world. In the past, the synchronous communication between services in the cluster was mainly HTTP/1.1. With the release of HTTP/2, many systems switch to HTTP/2 as a way of communicating between services, in order to improve transmission efficiency through persistent connections. However, the internal load balancer kube-proxy of Kubernetes is a network load balancer. It only handles the L4 communication protocol. While HTTP/2 implements persistent connection in L7, once the connection is established, there's no direct traffic to other Pods in the subsequent data transmission process. In order to solve this problem, many articles suggest using service mesh, which can balance traffic through sidecar proxy. This paper designed several experiments to compare the performance differences between Kubernetes with kube-proxy load balancer and Kubernetes with service mesh load balancer. Using service mesh to balance HTTP/2 traffic in Kubernetes can improve system efficiency only in suitable scenarios.

[1] Mao, Y., Fu, Y., Gu, S., Vhaduri, S., Cheng, L., & Liu, Q. “Resource management schemes for cloud-native platforms with computing containers of docker and kubernetes.” arXiv preprint arXiv:2010.10350, 2020.
[2] Rahman, J., & Lama, P. . “Predicting the end-to-end tail latency of containerized microservices in the cloud.” In 2019 IEEE International Conference on Cloud Engineering (IC2E), Prague, Czech Republic, Aug. 2019, pp. 200-210, doi: 10.1109/IC2E.2019.00034.
[3] Belshe, M., Peon, R., and M. Thomson, Ed. . “Hypertext Transfer Protocol Version 2 (HTTP/2).” RFC 7540. https://www.rfc-editor.org/info/rfc7540 (accessed June 20, 2022).
[4] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee. “Hypertext Transfer Protocol -- HTTP/1.1.” RFC 2616. https://www.rfc-editor.org/info/rfc2616 (accessed June 20, 2022).
[5] Jean de Klerk. “gRPC on HTTP/2 Engineering a Robust, High-performance Protocol.” gRPC. https://grpc.io/blog/grpc-on-http2/ (accessed June 20, 2022).
[6] “Load Balancer.” F5. https://www.f5.com/services/resources/glossary/load-balancer (accessed June 20, 2022).
[7] “Service.” Kubernetes. https://kubernetes.io/docs/concepts/services-networking/service/ (accessed June 20, 2022).
[8] William Morgan. “gRPC Load Balancing on Kubernetes without Tears.” Kubernetes. https://kubernetes.io/blog/2018/11/07/grpc-load-balancing-on-kubernetes-without-tears/ (accessed June 20, 2022).
[9] Daniele Polencic. “Load balancing and scaling long-lived connections in Kubernetes.” Learnk8s. https://learnk8s.io/kubernetes-long-lived-connections (accessed June 20, 2022).
[10] Li, W., Lemieux, Y., Gao, J., Zhao, Z., & Han, Y. . “Service mesh: Challenges, state of the art, and future research opportunities.” In 2019 IEEE International Conference on Service-Oriented System Engineering (SOSE), San Francisco, CA, USA, May 2019, pp. 122-1225, doi: 10.1109/SOSE.2019.00026.
[11] Alexander S. Gillis. “sidecar proxy.” TechTarget. https://www.techtarget.com/searchitoperations/definition/sidecar-proxy (accessed June 20, 2022).
[12] “Preventing Long Tail Latency.” Section. https://www.section.io/blog/preventing-long-tail-latency/ (accessed June 20, 2022).
[13] “Kubernetes Components.” Kubernetes. https://kubernetes.io/docs/concepts/overview/components/ (accessed June 20, 2022).
[14] Alex Pollitt. “Comparing kube-proxy modes: iptables or IPVS?” Tigera. https://www.tigera.io/blog/comparing-kube-proxy-modes-iptables-or-ipvs/ (accessed June 20, 2022).
[15] “Principles.” minikube. https://minikube.sigs.k8s.io/docs/contrib/principles/ (accessed June 20, 2022).
[16] “Performance and Scalability.” Istio. https://istio.io/latest/docs/ops/deployment/performance-and-scalability/ (accessed June 20, 2022).
[17] “minikube start.” minikube. https://minikube.sigs.k8s.io/docs/start/ (accessed June 20, 2022).
[18] “Introduction to Consul.” Consul. https://www.consul.io/docs/intro (accessed June 20, 2022).
[19] “Overview.” Linkerd. https://linkerd.io/2.12/overview/ (accessed June 20, 2022).
[20] Sachin Manpathak. “Kubernetes Service Mesh: A Comparison of Istio, Linkerd, and Consul.” Platform9. https://platform9.com/blog/kubernetes-service-mesh-a-comparison-of-istio-linkerd-and-consul/ (accessed June 20, 2022).
[21] Guillaume Dury. “A Kubernetes Service Mesh Comparison.” Toptal. https://www.toptal.com/kubernetes/service-mesh-comparison (accessed June 20, 2022).
[22] “Custom Resources.” Kubernetes. https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/ (accessed June 20, 2022).
[23] “Usage statistics of HTTP/2 for websites.” w3techs. https://w3techs.com/technologies/details/ce-http2 (accessed June 20, 2022).
[24] Bishop, M., Ed. . “HTTP/3.” RFC 9114. https://www.rfc-editor.org/rfc/rfc9114.html (accessed June 20, 2022).
[25] ABCOM. “HTTP/1.1 vs HTTP/2: What's the Difference?” DigitalOcean. https://www.digitalocean.com/community/tutorials/http-1-1-vs-http-2-what-s-the-difference (accessed June 20, 2022).
[26] “What is Locust?” Locust. https://docs.locust.io/en/stable/what-is-locust.html (accessed June 20, 2022).
[27] Ashok, Sachin, P. Brighten Godfrey, and Radhika Mittal. “Leveraging Service Meshes as a New Network Layer.” In Proceedings of the Twentieth ACM Workshop on Hot Topics in Networks, New York, NY, USA, Nov. 2021, pp. 229-236, doi: 10.1145/3484266.3487379.
[28] “What is Envoy.” Envoy. https://www.envoyproxy.io/docs/envoy/latest/intro/what_is_envoy (accessed June 20, 2022).
[29] N. Dattatreya Nadig, “Testing Resilience of Envoy Service Proxy with Microservices.” Master’s Thesis. KTH, School of Electrical Engineering and Computer Science, 2019.
[30] “SPOTIFY APOLLO OPERATIONS MANUAL.” Apollo. http://spotify.github.io/apollo/ (accessed June 20, 2022).