為因應物聯網需求，第三代合作夥伴計畫(3rd Generation Partnership Project, 3GPP)於Release 13中以Long Term Evolution (LTE)為基礎提出了窄頻物聯網(Narrowband Internet of Thing, NB-IoT)技術，其修改並簡化了LTE規格使其更符合物聯網裝置需求並可與現行系統並存，而在Release 14中更開放非錨載波(non-anchor carrier)支援隨機存取(Random Access)功能，如此可舒緩原UE只能透過錨載波進行隨機存取而造成網路壅塞之問題。
而原先非錨載波上行用於Narrowband Physical Uplink Shared Channel (NPUSCH)資料傳輸，當調度其支援於Narrowband Physical Random Access Channel (NPRACH)隨機存取將壓縮可傳輸之資源量，而當隨機存取資源不足將造成網路壅塞使UE無法建立radio resource control (RRC)連線上傳資料，因此eNB端如何配置非錨載波支援隨機存取而不造成資源浪費，使資源有效利用為一討論議題。
本論文提出Prediction based Random Access Resource Allocation scheme (PRARA)透過強化學習預測所需開放之資源，並以碰撞子載波(subcarrier)個數預測及動態開放二次競爭之資源，在資源許可的情況下有效使用資源並提升隨機存取之效能。

3rd Generation Partnership Project (3GPP) proposed NarrowBand Internet of Things (NB-IoT) based on Long Term Evolution (LTE) for IoT application in Release 13. It modifies and simplifies the LTE specification to let it be compatible with IoT devices and can coexist with existing LTE systems. In Release 14, the random access procedure can be supported in non-anchor carriers, which alleviate the problem that network congestion may occurs if UE can only random access via anchor carrier.
The non-anchor carriers in uplink are used for data transmission. However, if eNB schedules non-anchor carrier for Narrowband Physical Random Access Channel (NPRACH) then it also compresses the Narrowband Physical Uplink Shared Channel (NPUSCH) resource. When NPRACH resource is insufficient, which leads to network congestion, UEs might not be able to complete radio resource control (RRC) connection. So how to configure non-anchor carriers to support random access procedure without causing waste of resources is an importance issue.
In this thesis, we propose a Prediction based Random Access Resource Allocation scheme (PRARA), which firstly predicts the number of required resources based on reinforcement learning, and secondly, dynamically allocates the number of secondary contention resources according to the number of collided subcarriers. We aim to increase the performance and resource efficiency of random access in condition of limited resource.

[1] Y.-P. E. Wang, X. Lin, A. Adhikary, A. Grovlen, Y. Sui, Y. Blankenship, J. Bergman and H. S. Razaghi, "A Primer on 3GPP Narrowband Internet of Things," IEEE Communications Magazine, vol. 55, pp. 117-123, 2017.
[2] [Online]. Available: https://www.2cm.com.tw/2cm/zh-tw/magazine/-Technology/64BA704E800148C781D35F664663A5B4. [Accessed 23 5 2019].
[3] B. V. N. M. A. G. Rapeepat Ratasuk, "NB-IoT System for M2M Communication," in Wireless Communications and Networking Conference (WCNC), pp. 428–432.,Aug. 2016.
[4] L. Feltrin, G. Tsoukaneri, M. Condoluci, C. Buratti, T. Mahmoodi, M. Dohler and R. Verdone, "Narrowband IoT: A Survey on Downlink and Uplink Perspectives," IEEE Wireless Communications ( Volume: 26 , Issue: 1 ), vol. 26, pp. 78-86, February 2019.
[5] [Online]. Available: http://www.sharetechnote.com/html/Handbook_LTE_NB_FrameStructure_DL.html. [Accessed 31 5 2019].
[6] [Online]. Available: http://www.sharetechnote.com/html/Handbook_LTE_NB_MultiCarrier.html. [Accessed 25 5 2019].
[7] "Narrowband Internet of Things Whitepaper," [Online]. Available: https://cdn.rohde-schwarz.com/pws/dl_downloads/dl_application/application_notes/1ma266/1MA266_0e_NB_IoT.pdf. [Accessed 3 6 2019].
[8] X. Lin, A. Adhikary and Y.-P. E. Wang, "Random Access Preamble Design and Detection for 3GPP Narrowband IoT Systems," IEEE Wireless Communications Letters ( Volume: 5 , Issue: 6 ), vol. 5, pp. 640-643, 12 2016.
[9] D. Pandey and P. Pandey, "Approximate Q-Learning: An Introduction," in 2010 Second International Conference on Machine Learning and Computing, Bangalore, India, 2010.
[10] J. Li, Z. Zhao and R. Li, "Machine learning-based IDS for software-defined 5G network," IET Networks, vol. 7, no. 2, pp. 53-60, 1 3 2018.
[11] K. Lee and J. W. Jang, "An Efficient Contention Resolution Scheme for Massive IoT Devices in Random Access to LTE-A Networks," IEEE Access ( Volume: 6 ), pp. 67118 - 67130, 11 2018.
[12] C. Di, B. Zhang, Q. Liang, S. Li and Y. Guo, "Learning Automata based Access Class Barring Scheme for Massive Random Access in Machine-to-Machine Communications," IEEE Internet of Things Journal ( Early Access ), 2019.
[13] Z. Chen and D. B. Smith, "Heterogeneous Machine-Type Communications in Cellular Networks: Random Access Optimization by Deep Reinforcement Learning," in IEEE International Conference on Communications (ICC), Kansas City, MO, USA, 2018.
[14] N. Jiang, Y. Deng, A. Nallanathan and J. A. Chambers, "Reinforcement Learning for Real-Time Optimization in NB-IoT Networks," IEEE Journal on Selected Areas in Communications ( Volume: 37 , Issue: 6 ), pp. 1424 - 1440, June 2019.
[15] Y. Zhao, K. Liu, H. Yan and L. Huang, "A classification back-off method for capacity optimization in NB-IOT random access," in 2017 11th IEEE International Conference on Anti-counterfeiting, Security, and Identification (ASID), Xiamen, China, 2017.
[16] R. Ratasuk, N. Mangalvedhe, Z. Xiong, M. Robert and D. Bhatoolaul, "Enhancements of narrowband IoT in 3GPP Rel-14 and Rel-15," in 2017 IEEE Conference on Standards for Communications and Networking (CSCN), Helsinki, Finland, 2017.
[17] 3GPP TR 21.914, "Release 14 Description ;Summary of Rel-14 Work Items," V14.0.0, May 2018.
[18] W. S. Jeon, S. B. Seo and D. G. Jeong, "Effective Frequency Hopping Pattern for ToA Estimation in NB-IoT Random Access," IEEE Transactions on Vehicular Technology ( Volume: 67 , Issue: 10 , Oct. 2018 ), pp. 10150 - 10154, 10 2018.
[19] G.-Y. Xue, "Study of Random Access Scheme and Its Resource Adaptive Allocation in NB-IoT Network," National Central University, thesis, 2018.
[20] 3GPP TR 37.868, "Study on RAN Improvements for Machine-type Communications".
[21] "Deep Deterministic Policy Gradient," [Online]. Available: https://spinningup.openai.com/en/latest/algorithms/ddpg.html#id1. [Accessed 19 7 2019].
[22] A. P. B. M. G. H. P. L. S. K. Volodymyr Mnih, "Asynchronous Methods for Deep Reinforcement Learning," 2016.