近幾年第五代行動通訊（5G）蓬勃發展，國際電信聯盟（International Telecommunication Union, ITU）將5G主要規範為三大應用場景，包含增強型行動頻寬（Enhanced Mobile Broadband, eMBB）、超可靠度和低延遲通訊（Ultra-reliable and low latency communications, URLLC）以及大規模機器通訊（Massive machine type communications, mMTC），其中URLLC可應用在車聯網等強調低延遲高可靠的即時傳輸，mMTC可應用在智能城市等大量物聯網設備的環境，由於網路資源數量有限，且兩者要求目標不同，在目前通訊網路系統中要如何讓這兩種不同類型的設備達到更好的性能有著巨大的挑戰。
上行業務以稀疏代碼多重連接（Sparse Code Multiple Access, SCMA）的場景下，將用戶設備（User Equipment, UE）選取競爭傳輸單元（Contention Transmission Unit, CTU），並採無允諾上行（Uplink Grant-free）方式來降低基地台（Base Station, BS）與UE間的授權延遲。此外，會基於mapping rule分配方式競爭選取CTU，本論文為了滿足URLLC低延遲及高可靠的特性提出了步移增強式mapping rule（Enhanced Mapping Rule with Step Movement, EMRSM），為了使mMTC更有效率的使用資源提出了多階段之CTU分配（Multi-stage CTU Allocation, MCA），並利用強化式學習動態調整CTU資源來探討URLLC及mMTC的共存方案。從模擬結果可看出，EMRSM可滿足URLLC低延遲高可靠的要求，利用MCA可看出大量的封包訪問網路不會造成壅塞，將強化式學習動態調整CTU資源用在mapping rule分配方法下，可些微提升成功率，然而由於EMRSM成功率很高，因此強化式學習動態調整CTU資源用在EMRSM分配方法下，反而降低了成功率。

In recent years, the fifth generation of mobile communications (5G) has flourished. The International Telecommunication Union (ITU) has standardized 5G into three major application scenarios, including eMBB, URLLC and mMTC. URLLC can be applied to the Internet of Vehicles and other instant transmissions that emphasize low latency and high reliability. The mMTC can be applied to the environment of a large number of Internet of Things devices such as smart cities. However, the number of network resources is limited, and the requirements of the two are different. How to coexist these two types of equipment with different requirements is a huge challenge to the current communication network system.
In SCMA transmission, Contention Transmission Unit (CTU) is selected by UE. The Uplink Grant-free method is adopted to reduce the authorization delay between BS and UE. In addition, the CTU will be selected based on the mapping rule allocation method. This paper proposes Enhanced Mapping Rule with Step Movement (EMRSM) in order to meet the low-latency and high-reliability characteristics of URLLC. Multi-stage CTU Allocation (MCA) is proposed in order to make mMTC use resources more efficiently. And using reinforcement learning to dynamically adjust CTU resources to explore the coexistence of URLLC and mMTC. According to the simulation results that EMRSM can meet the requirements of URLLC for low latency and high reliability. A large number of packets accessing the network will not cause congestion by MCA. Dynamic adjustment of CTU resources with reinforcement learning for mapping rule can slightly increase the success rate. However, dynamic adjustment of CTU resources with reinforcement learning for EMRSM actually reduces the success rate because the EMRSM success rate is quite high.

[1] ITU-R, "M.2083 : IMT Vision - Framework and overall objectives of the future development of IMT for 2020 and beyond," 9 2015. [Online]. Available: https://www.itu.int/dms_pubrec/itu-r/rec/m/R-REC-M.2083-0-201509-I!!PDF-E.pdf. [Accessed 6 2020].
[2] Kelvin Au, Liqing Zhang, Hosein Nikopour, Eric Yi, Alireza Bayesteh, Usa Vilaipornsawai, Jianglei Ma, and Peiying Zhu, "Uplink Contention Based SCMA for 5G Radio Access," IEEE Globecom Workshops (GC Wkshps), 2014.
[3] 3GPP TR 38.913, “Study on Scenarios and Requirements for Next Generation Access Technologies,” 3GPP Tech. Rep., V16.0.0, Nov. 2020.
[4] 3GPP TS 22.261, “Service requirements for the 5G system,” 3GPP Tech. Rep., V15.8.0, Sep. 2019.
[5] [Online]. Available: https://www.nttdocomo.co.jp/english/binary/pdf/corporate/technology/rd/technical_journal/bn/vol19_3/vol19_3_003en.pdf.
[6] 3GPP TS 36.211, “Evolved Universal Terrestrial Radio Access (E-UTRA);Physical channels and modulation,” 3GPP Tech. Rep., V16.5.0, Mar. 2021.
[7] 3GPP TS 38.211, “NR;Physical channels and modulation,” 3GPP Tech. Rep., V16.5.0, Mar. 2021.
[8] [Online]. Available: https://www.sharetechnote.com/html/5G/5G_FrameStructure.html.
[9] Luciano Miuccio, Daniela Panno, and Salvatore Riolo, " Joint Control of Random Access and Dynamic Uplink Resource Dimensioning for Massive MTC in 5G NR Based on SCMA," IEEE Internet of Things Journal, vol. 7, no. 6, June, 2020.
[10] [Online]. Available: https://www.sharetechnote.com/html/RACH_LTE.html.
[11] 3GPP TS 38.331, “NR; Radio Resource Control (RRC); Protocol specification,” 3GPP Tech. Rep., V16.4.1, Mar. 2021.
[12] Seokjae Moon and Jang-Won Lee, " Integrated Grant-Free Scheme for URLLC and mMTC," IEEE 5G World Forum (5GWF), 2020.
[13] Yan Liu, Yansha Deng, Maged Elkashlan, Arumugam Nallanathan, and George K. Karagiannidis, "Analyzing Grant-Free Access for URLLC Service," IEEE Journal on Selected Areas in Communications, vol. 39, no. 3, March, 2021.
[14] [Online]. Available: https://www.intel.la/content/www/xl/es/artificial-intelligence/posts/difference-between-ai-machine-learning-deep-learning.html.
[15] [Online]. Available: https://towardsdatascience.com/deep-q-network-dqn-i-bce08bdf2af.
[16] [Online]. Available: https://medium.com/%E9%9B%9E%E9%9B%9E%E8%88%87%E5%85%94%E5%85%94%E7%9A%84%E5%B7%A5%E7%A8%8B%E4%B8%96%E7%95%8C/%E6%A9%9F%E5%99%A8%E5%AD%B8%E7%BF%92-ml-note-reinforcement-learning-%E5%BC%B7%E5%8C%96%E5%AD%B8%E7%BF%92-dqn-%E5%AF%A6%E4%BD%9Catari-game-7f9185f.
[17] Jiali Shen, Wen Chen, Fan Wei and Yongpeng Wu, "ACK Feedback based UE-to-CTU Mapping Rule for SCMA Uplink Grant-Free Transmission," International Conference on Wireless Communications and Signal Processing (WCSP), 2017.
[18] Shuai Han, Xiangxue Tai, Weixiao Meng, and Cheng Li, "A Resource Scheduling Scheme Based on Feed-Back for SCMA Grant-Free Uplink Transmission," IEEE International Conference on Communications (ICC), 2017.
[19] Trung-Kien Le, Umer Salim, and Florian Kaltenberger, "Enhancing URLLC Uplink Configured-grant Transmissions," IEEE Vehicular Technology Conference (VTC), 2021.
[20] Ziqi Chen and David B. Smith, "Heterogeneous Machine-Type Communications in Cellular Networks: Random Access Optimization by Deep Reinforcement Learning," IEEE International Conference on Communications (ICC), 2018.
[21] Jiazhen Zhang, Xiaofeng Tao, Huici Wu, Ning Zhang, and Xuefei Zhang, " Deep Reinforcement Learning for Throughput Improvement of the Uplink Grant-Free NOMA System," IEEE Internet of Things Journal, vol. 7, no. 7, Jul, 2020.