無線混合現實(MR)系統，需要提供使用者及時且準確的與虛擬物件互動
的沉浸式體驗。為了在有限的硬體計算資源下達成此目標，現有的方式
為將計算轉移至邊緣伺服器，並藉由預測的方式減緩MTP延遲所造成看
到的畫面不是當下畫面的情況發生，提前渲染視野的影像進行傳輸。由
於預測不是無錯誤的，因此可能出現物件顯示位置錯誤的問題，而導致
使用者無法準確與物件進行互動。為了解決這個問題，我們提出了使用
本地端的物件精準位置做為定位輔助的串流架構並且開發了基於所提出
架構的串流資料優化算法，避免多餘的資料傳輸。實驗結果顯示，在相
同的MTP延遲下，所提出的串流架構在虛擬物件定位和網路資料量傳輸
方面都優於現有的方法。

Wireless mixed reality (MR) systems must provide users with an immersive experience of interacting with virtual objects in real-time and accurately. In order to achieve this goal under limited hardware computing resources. The existing method is to transfer the calculation to the edge server and use the field of view (FoV) prediction method to reduce the situation that the image seen is not the current image caused by the Motion-to-Photon (MTP) latency and render the relative video in advance for transmission. Since predictions are not error-free, objects may be displayed in the wrong position, preventing users from interacting with objects inaccurately. To solve this problem, we propose a streaming architecture that uses the absolute position of objects on the MR devices as a positioning aid. Then, we develop streaming data optimization algorithms based on the proposed architecture to avoid redundant data transmission. Experimental results show that under the same MTP latency, the proposed streaming architecture outperforms existing methods in both virtual object location and network data volume transmission.

[1] P. Milgram and F. Kishino, “A taxonomy of mixed reality visual displays,” IEICE
TRANSACTIONS on Information and Systems, vol. 77, no. 12, pp. 1321–1329, 1994.
[2] O. Schreer, I. Feldmann, P. Kauff, P. Eisert, D. Tatzelt, C. Hellge, K. M ̈uller,
S. Bliedung, and T. Ebner, “Lessons learned during one year of commercial vol-
umetric video production,” SMPTE Motion Imaging Journal, vol. 129, no. 9, pp.
31–37, 2020.
[3] S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, P. A. Chou, R. A. Co-
hen, M. Krivoku ́ca, S. Lasserre, Z. Li et al., “Emerging mpeg standards for point
cloud compression,” IEEE Journal on Emerging and Selected Topics in Circuits and
Systems, vol. 9, no. 1, pp. 133–148, 2018.
[4] A. Clemm, M. T. Vega, H. K. Ravuri, T. Wauters, and F. De Turck, “Toward truly
immersive holographic-type communication: Challenges and solutions,” IEEE Com-
munications Magazine, vol. 58, no. 1, pp. 93–99, 2020.
[5] S. Shi and C.-H. Hsu, “A survey of interactive remote rendering systems,” ACM
Computing Surveys (CSUR), vol. 47, no. 4, pp. 1–29, 2015.
[6] J. Zhao, R. S. Allison, M. Vinnikov, and S. Jennings, “Estimating the motion-
to-photon latency in head mounted displays,” in 2017 IEEE Virtual Reality (VR).
IEEE, 2017, pp. 313–314.
[7] E. Vikberg et al., “Optimizing webrtc for cloud streaming of xr,” 2021.
[8] P. Eisert and P. Fechteler, “Low delay streaming of computer graphics,” in 2008 15th
IEEE International Conference on Image Processing. IEEE, 2008, pp. 2704–2707.
[9] I. Nave, H. David, A. Shani, Y. Tzruya, A. Laikari, P. Eisert, and P. Fechteler,
“Games@ large graphics streaming architecture,” in 2008 IEEE International Sym-
posium on Consumer Electronics. IEEE, 2008, pp. 1–4.
[10] M. A. Livingston and Z. Ai, “The effect of registration error on tracking distant
augmented objects,” in 2008 7th IEEE/ACM International Symposium on Mixed and
Augmented Reality. IEEE, 2008, pp. 77–86.
[11] S. G ̈ul, S. Bosse, D. Podborski, T. Schierl, and C. Hellge, “Kalman filter-based head
motion prediction for cloud-based mixed reality,” in Proceedings of the 28th ACM
International Conference on Multimedia, 2020, pp. 3632–3641.
[12] R. K. Kundu, A. Rahman, and S. Paul, “A study on sensor system latency in vr
motion sickness,” Journal of Sensor and Actuator Networks, vol. 10, no. 3, p. 53,
2021.
[13] X. Hou and S. Dey, “Motion prediction and pre-rendering at the edge to enable ultra-
low latency mobile 6dof experiences,” IEEE Open Journal of the Communications
Society, vol. 1, pp. 1674–1690, 2020.
[14] S. G ̈ul, D. Podborski, J. Son, G. S. Bhullar, T. Buchholz, T. Schierl, and C. Hellge,
“Cloud rendering-based volumetric video streaming system for mixed reality ser-
vices,” in Proceedings of the 11th ACM multimedia systems conference, 2020, pp.
357–360.
[15] K. Lee, J. Yi, Y. Lee, S. Choi, and Y. M. Kim, “Groot: a real-time streaming system
of high-fidelity volumetric videos,” in Proceedings of the 26th Annual International
Conference on Mobile Computing and Networking, 2020, pp. 1–14.
[16] R. B. Rusu and S. Cousins, “3d is here: Point cloud library (pcl),” in 2011 IEEE
international conference on robotics and automation. IEEE, 2011, pp. 1–4.
[17] P. Alliez, D. Cohen-Steiner, O. Devillers, B. L ́evy, and M. Desbrun, “Anisotropic
polygonal remeshing,” in ACM SIGGRAPH 2003 Papers, 2003, pp. 485–493.
[18] E. Zerman, C. Ozcinar, P. Gao, and A. Smolic, “Textured mesh vs coloured point
cloud: A subjective study for volumetric video compression,” in 2020 Twelfth Inter-
national Conference on Quality of Multimedia Experience (QoMEX). IEEE, 2020,
pp. 1–6.
[19] T. K ̈am ̈ar ̈ainen, M. Siekkinen, J. Eerik ̈ainen, and A. Yl ̈a-J ̈a ̈aski, “Cloudvr: Cloud
accelerated interactive mobile virtual reality,” in Proceedings of the 26th ACM inter-
national conference on Multimedia, 2018, pp. 1181–1189.
[20] L. Liu, H. Li, and M. Gruteser, “Edge assisted real-time object detection for mobile
augmented reality,” in The 25th annual international conference on mobile comput-
ing and networking, 2019, pp. 1–16.
[21] Z. Long, H. Dong, and A. El Saddik, “Interacting with new york city data by hololens
through remote rendering,” IEEE Consumer Electronics Magazine, 2022.
[22] J. Park, P. A. Chou, and J.-N. Hwang, “Rate-utility optimized streaming of volumet-
ric media for augmented reality,” IEEE Journal on Emerging and Selected Topics in
Circuits and Systems, vol. 9, no. 1, pp. 149–162, 2019.
[23] S. Liu, X. Xu, and M. Claypool, “A survey and taxonomy of latency compensation
techniques for network computer games,” ACM Computing Surveys (CSUR), 2022.
[24] S. Yoon, H. jeong Lim, J. H. Kim, H.-S. Lee, Y.-T. Kim, and S. Sull, “Deep 6-dof
head motion prediction for latency in lightweight augmented reality glasses,” in 2022
IEEE International Conference on Consumer Electronics (ICCE). IEEE, 2022, pp.
1–6.
[25] I. Telecom et al., “Advanced video coding for generic audiovisual services,” ITU-T
Recommendation H. 264, 2003.
[26] M. Wien, “High efficiency video coding,” Coding Tools and specification, vol. 24,
2015.
[27] E. Cuervo, A. Wolman, L. P. Cox, K. Lebeck, A. Razeen, S. Saroiu, and M. Musu-
vathi, “Kahawai: High-quality mobile gaming using gpu offload,” in Proceedings
of the 13th Annual International Conference on Mobile Systems, Applications, and
Services, 2015, pp. 121–135.
[28] “4dviews volumetric motion capture systems.” https://www.4dviews.com/
volumetric-resources.
[29] S. G ̈ul, S. Bosse, D. Podborski, T. Schierl, C. Hellge, M. A. Kastner, and J. Zah ́alka,
“Reproducibility companion paper: Kalman filter-based head motion prediction for
cloud-based mixed reality,” in Proceedings of the 29th ACM International Confer-
ence on Multimedia, 2021, pp. 3619–3621.