隨著科技快速的發展下，增強現實（AR）、虛擬現實（VR）和混合現實（MR）等技術成為一個備受關注的領域。然而這些技術在遠端算繪應用中仍然需要面臨一些困難和問題，例如延遲和預測不準確等。為了防止出現這些問題，本文提出了一種多人遠端算繪架構，目標於提供良好的MR應用體驗。該架構利用伺服器端物件串流相機設計和本地端定位輔助，幾乎消除了延遲帶來的不順暢物件位移現象。同時，通過最佳化網路流量，考慮「人與物件」之間的關係，透過動態調整解析度以及視野剔除或合併物件等方法，提高了網路使用效率和提升用戶體驗。實驗結果顯示，該架構的數據最佳化方法平均降低了80%的資料量，與傳統的MR串流架構相比減少了70%。此外，該架構還支持多個用戶使用不同廠牌的頭戴顯示設備或行動設備加入虛擬空間並進行物件互動，並減少了伺服器上的硬體消耗。

In the wave of rapid development of modern technology, augmented reality (AR), virtual reality (VR), and mixed reality (MR) have become prominent fields. However, these technologies still face some challenges in remote rendering applications, such as latency and inaccurate prediction. To address these issues, this paper proposes a multi-user remote rendering architecture aimed at providing users with a high-quality MR application experience.
The architecture utilizes a server-side object streaming camera design and local positioning assistance to virtually eliminate the noticeable object displacement caused by latency. Simultaneously, by optimizing network traffic, considering the relationship between users and objects, dynamically adjusting resolutions, and performing view frustum culling or object merging, the network efficiency is improved, and the user experience is enhanced.
Experimental results demonstrate that the data optimization method of this architecture reduces the average data volume by 80%, compared to a traditional MR streaming architecture, which reduces the data volume by 70%. Additionally, this architecture supports multiple users using different brands of head-mounted displays or mobile devices to enter the virtual space and interact with objects, while reducing hardware consumption on the server.

[1] S. Orts-Escolano, C. Rhemann, S. Fanello, W. Chang, A. Kowdle, Y. Degtyarev,
D. Kim, P. L. Davidson, S. Khamis, M. Dou et al., “Holoportation: Virtual 3d
teleportation in real-time,” in Proceedings of the 29th annual symposium on user
interface software and technology, 2016, pp. 741–754.
[2] S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, P. A. Chou, R. A. Co-
hen, M. Krivoku ́ca, S. Lasserre, Z. Li et al., “Emerging mpeg standards for point
cloud compression,” IEEE Journal on Emerging and Selected Topics in Circuits and
Systems, vol. 9, no. 1, pp. 133–148, 2018.
[3] L. Song, A. Chen, Z. Li, Z. Chen, L. Chen, J. Yuan, Y. Xu, and A. Geiger, “Nerf-
player: A streamable dynamic scene representation with decomposed neural radi-
ance fields,” IEEE Transactions on Visualization and Computer Graphics, vol. 29,
no. 5, pp. 2732–2742, 2023.
[4] S. Shi and C.-H. Hsu, “A survey of interactive remote rendering systems,” ACM
Computing Surveys (CSUR), vol. 47, no. 4, pp. 1–29, 2015.
[5] S. G ̈ul, D. Podborski, J. Son, G. S. Bhullar, T. Buchholz, T. Schierl, and C. Hellge,
“Cloud rendering-based volumetric video streaming system for mixed reality ser-
vices,” in Proceedings of the 11th ACM multimedia systems conference, 2020, pp.
357–360.
[6] J. Zhao, R. S. Allison, M. Vinnikov, and S. Jennings, “Estimating the motion-
to-photon latency in head mounted displays,” in 2017 IEEE Virtual Reality (VR).
IEEE, 2017, pp. 313–314.
[7] R. T. Azuma, “Predictive tracking for augmented reality,” Ph.D. dissertation, Uni-
versity of North Carolina at Chapel Hill, 1995.
[8] M. A. Livingston and Z. Ai, “The effect of registration error on tracking distant
augmented objects,” in 2008 7th IEEE/ACM International Symposium on Mixed and
Augmented Reality. IEEE, 2008, pp. 77–86.
[9] S. G ̈ul, S. Bosse, D. Podborski, T. Schierl, and C. Hellge, “Kalman filter-based head
motion prediction for cloud-based mixed reality,” in Proceedings of the 28th ACM
International Conference on Multimedia, 2020, pp. 3632–3641.
[10] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, “So-
cial lstm: Human trajectory prediction in crowded spaces,” in Proceedings of the
IEEE conference on computer vision and pattern recognition, 2016, pp. 961–971.
[11] X. Hou and S. Dey, “Motion prediction and pre-rendering at the edge to enable ultra-
low latency mobile 6dof experiences,” IEEE Open Journal of the Communications
Society, vol. 1, pp. 1674–1690, 2020.
[12] S.-P. Chuah, C. Yuen, and N.-M. Cheung, “Cloud gaming: a green solution to mas-
sive multiplayer online games,” IEEE Wireless Communications, vol. 21, no. 4, pp.
78–87, 2014.
[13] T. K ̈am ̈ar ̈ainen, M. Siekkinen, J. Eerik ̈ainen, and A. Yl ̈a-J ̈a ̈aski, “Cloudvr: Cloud
accelerated interactive mobile virtual reality,” in Proceedings of the 26th ACM inter-
national conference on Multimedia, 2018, pp. 1181–1189.
[14] Z. Long, H. Dong, and A. El Saddik, “Interacting with new york city data by hololens
through remote rendering,” IEEE Consumer Electronics Magazine, vol. 11, no. 5, pp.
64–72, 2022.
[15] S. Shi, K. Nahrstedt, and R. Campbell, “A real-time remote rendering system for
interactive mobile graphics,” ACM Transactions on Multimedia Computing, Com-
munications, and Applications (TOMM), vol. 8, no. 3s, pp. 1–20, 2012.
[16] D. C. Barboza, D. C. Muchaluat-Saade, E. W. G. Clua, and D. G. Passos, “An ar-
chitecture for multi-layer object coding in 2d game streaming using shared data in a
multi-user environment,” Entertainment Computing, vol. 42, p. 100490, 2022.
[17] S. Liu, X. Xu, and M. Claypool, “A survey and taxonomy of latency compensation
techniques for network computer games,” ACM Computing Surveys (CSUR), 2022.
[18] S. G ̈ul, D. Podborski, T. Buchholz, T. Schierl, and C. Hellge, “Low-latency cloud-
based volumetric video streaming using head motion prediction,” in Proceedings of
the 30th ACM Workshop on Network and Operating Systems Support for Digital
Audio and Video, 2020, pp. 27–33.
[19] S. Yoon, H. jeong Lim, J. H. Kim, H.-S. Lee, Y.-T. Kim, and S. Sull, “Deep 6-dof
head motion prediction for latency in lightweight augmented reality glasses,” in 2022
IEEE International Conference on Consumer Electronics (ICCE). IEEE, 2022, pp.
1–6.
[20] H.-L. Zeng, “Larr:delay compensation via local positioning for mr remote render-
ing,” 6 2022.
[21] M. Ester, H.-P. Kriegel, J. Sander, X. Xu et al., “A density-based algorithm for dis-
covering clusters in large spatial databases with noise.” in kdd, vol. 96, no. 34, 1996,
pp. 226–231.
[22] “4dviews volumetric motion capture systems.” https://www.4dviews.com/
volumetric-resources.
[23] S. G ̈ul, S. Bosse, D. Podborski, T. Schierl, C. Hellge, M. A. Kastner, and J. Zah ́alka,
“Reproducibility companion paper: Kalman filter-based head motion prediction for
cloud-based mixed reality,” in Proceedings of the 29th ACM International Confer-
ence on Multimedia, 2021, pp. 3619–3621.