隨著移動網路的爆炸性成長，現行的基地台架構將難以負擔未來的流量需求。其中一個解決方式是廣建小型基地台，將大型基地台的流量分流至小型基地台實現負載均衡。
然而，移動網路的流量需求是隨時間改變的，這將造成在不需要的時段開啟過多閒置的小型基地台，因此需要一套機制台控制小型基地台的部屬與使用。
在考量能源效率下，作者提出基於深度增強式學習並結合流量預測以預先反應的角度來解決小型基地台部署時的能源問題。
流量負載平衡框架由環境模型、流量模型和決策模型組成。
整個負載平衡環境是建立在多個大型基地台下覆蓋多個小型基地台。
流量預測模型會根據歷史的流量狀況預測下個時間流量的最大值、平均值和最小值達到多任務聯合學習的優勢。
作者調查並研究了深度學習的各種方法在流量預測上的表現，這些方法包括了 CNN、3D CNN 和 CNN與RNN的結合。
在決策模型上，根據流量預測模型的輸出結果，使用了深度增強式學習進行決策，要預先開啟多少數量的小型基地台才能達到最佳的能源使用效率。
最後，基於真實世界流量資料的實驗證明，隨著移動流量需求快速增長至超出一大型基地台的負荷，從流量預測的角度輔助決策可以更好的平衡大小型基地台間的負載。

With the explosive growth in demand for mobile traffic, one of the promising solutions is to offload mobile traffic to small cells.
However, mobile traffic is time-varying which will cause large numbers of small cells were turned on at an unnecessary period.
In consideration of the energy problem, the author proposed a deep reinforcement learning based mobile offloading architecture with traffic prediction to solve the problem in a proactive manner.
The offloading architecture is composed of three components, environment, traffic prediction model, and decision model.
The environment comprises multiple macro cells with numerous small cells under their converge to offload mobile traffic.
The traffic prediction model is a multi-task learning architecture which can learn next epoch's maximum, average, and minimum mobile traffic at the same time.
The author studied multiple popular deep learning approaches, including RNN, 3D CNN, and the combination of CNN and RNN and examined what kind of structure would obtain better prediction accuracy in time series data set, realistic telecommunication data.
And in the decision model, the author implemented a deep Q network which takes charge of how many small cells should be turned on among a macro cell according to the prediction result coming from the traffic prediction model.
The experiments were conducted on realistic mobile data to prove the mobile traffic prediction is beneficial to offloading policy when the traffic demand has skyrocketed.

[1] Andrej Karpathy, “Cs231n, convolutional neural networks for visual recognition.” [Online]. Available: http://cs231n.github.io/assets/nn1/neural net2.jpeg
[2] LISA lab, “Convolutional neural networks (lenet).” [Online]. Available: http: //deeplearning.net/tutorial/ images/mylenet.png
[3] Denny Britz , “Recurrent neural networks tutorial, part 1 – introduction to rnns.” [Online]. Available: http://d3kbpzbmcynnmx.cloudfront.net/wp-content/uploads/ 2015/09/rnn.jpg
[4] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press Cambridge, 1998, vol. 1, no. 1.
[5] A. Nair, P. Srinivasan, S. Blackwell, C. Alcicek, R. Fearon, A. De Maria, V. Panneershelvam, M. Suleyman, C. Beattie, S. Petersen, S. Legg, V. Mnih, K. Kavukcuoglu, and D. Silver, “Massively Parallel Methods for Deep Reinforcement Learning,” arXiv, p. 14, 2015. [Online]. Available: http://arxiv.org/abs/1507.04296
[6] X. Chen, J. Wu, Y. Cai, H. Zhang, and T. Chen, “Energy-efficiency oriented traf- fic offloading in wireless networks: A brief survey and a learning approach for het- erogeneous cellular networks,” IEEE Journal on Selected Areas in Communications, vol. 33, no. 4, pp. 627–640, 2015.
[7] P. Sankt, “Lte networks: How far are the achievable capacities from the theoretical ones?” ICUMT2012, 2012.
[8] MARK HEATH, “Lte performance is good, but it’s not that good!” 2010. [Online]. Available: https://http://www.unwiredinsight.com/2010/lte-performance
[9] C. V. N. I. Cisco, “Global mobile data traffic forecast update, 2016–2021,” white paper, 2016.
[10] A. Aijaz, H. Aghvami, and M. Amani, “A survey on mobile data offloading: technical and business perspectives,” IEEE Wireless Communications, vol. 20, no. 2, pp. 104– 112, 2013.
[11] D. Lo ́pez-Pe ́rez, M. Ding, H. Claussen, and A. H. Jafari, “Towards 1 gbps/ue in cel- lular systems: Understanding ultra-dense small cell deployments,” IEEE Communi- cations Surveys & Tutorials, vol. 17, no. 4, pp. 2078–2101, 2015.
[12] X. Ge, S. Tu, G. Mao, C.-X. Wang, and T. Han, “5g ultra-dense cellular networks,” IEEE Wireless Communications, vol. 23, no. 1, pp. 72–79, 2016.
[13] J. Xu, J. Wang, Y. Zhu, Y. Yang, X. Zheng, S. Wang, L. Liu, K. Horneman, and Y. Teng, “Cooperative distributed optimization for the hyper-dense small cell deploy- ment,” IEEE Communications Magazine, vol. 52, no. 5, pp. 61–67, 2014.
[14] A. Imran and A. Zoha, “Challenges in 5G: how to empower SON with big data for enabling 5G,” IEEE Network, vol. 28, no. 6, pp. 27–33, Nov. 2014.
[15] M. Z. Shafiq, L. Ji, A. X. Liu, and J. Wang, “Characterizing and modeling internet traffic dynamics of cellular devices,” ACM SIGMETRICS Performance Evaluation Review, vol. 39, no. 1, p. 265, 2011.
[16] Y.Lv,Y.Duan,W.Kang,Z.Li,andF.-Y.Wang,“Trafficflowpredictionwithbigdata: a deep learning approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 2, pp. 865–873, 2015.
[17] T. P. Oliveira, J. S. Barbar, and A. S. Soares, “Computer network traffic prediction: a comparison between traditional and deep learning neural networks,” International Journal of Big Data Intelligence, vol. 3, no. 1, pp. 28–37, 2016.
[18] “Multi-scale Internet traffic forecasting using neural networks and time series meth- ods,” Expert Systems, vol. 29, no. 2, pp. 143–155, 2012.
[19] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.
[20] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level con- trol through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015.
[21] M. A. Marsan, L. Chiaraviglio, D. Ciullo, and M. Meo, “Multiple daily base station switch-offs in cellular networks,” in Communications and Electronics (ICCE), 2012 Fourth International Conference on. IEEE, 2012, pp. 245–250.
[22] C. Liu, B. Natarajan, and H. Xia, “Small cell base station sleep strategies for energy efficiency,” IEEE Transactions on Vehicular Technology, vol. 65, no. 3, pp. 1652– 1661, 2016.
[23] Y. S. Soh, T. Q. Quek, M. Kountouris, and H. Shin, “Energy efficient heterogeneous cellular networks,” IEEE Journal on Selected Areas in Communications, vol. 31, no. 5, pp. 840–850, 2013.
[24] V. A. Siris and M. Anagnostopoulou, “Performance and energy efficiency of mo- bile data offloading with mobility prediction and prefetching,” in World of Wireless, Mobile and Multimedia Networks (WoWMoM), 2013 IEEE 14th International Sym- posium and Workshops on a. IEEE, 2013, pp. 1–6.
[25] F. Xu, Y. Lin, J. Huang, D. Wu, H. Shi, J. Song, and Y. Li, “Big Data Driven Mobile Traffic Understanding and Forecasting: A Time Series Approach,” IEEE Transactions on Services Computing, vol. 9, no. 5, pp. 796–805, Sep. 2016.
[26] X. Zhou, Z. Zhao, R. Li, Y. Zhou, and H. Zhang, “The predictability of cellular net- works traffic,” 2012 International Symposium on Communications and Information Technologies, ISCIT 2012, pp. 973–978, 2012.
[27] R. Li, Z. Zhao, J. Zheng, Y. Chen, C. Mei, Y. Cai, and H. Zhang, “The Learning and Prediction of Application-level Traffic Data in Cellular Networks,” pp. 1–12, 2016. [Online]. Available: http://arxiv.org/abs/1606.04778
[28] W. Huang, G. Song, H. Hong, and K. Xie, “Deep architecture for traffic flow predic- tion: Deep belief networks with multitask learning,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 5, pp. 2191–2201, 2014.
[29] T. P. Oliveira, J. S. Barbar, and A. S. Soares, “Computer network traffic prediction: a comparison between traditional and deep learning neural networks,” International Journal of Big Data Intelligence, vol. 3, no. 1, p. 28, 2016. [Online]. Available: http://www.inderscience.com/link.php?id=73903
[30] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” California Univ San Diego La Jolla Inst for Cognitive Science, Tech. Rep., 1985.
[31] Y. LeCun, B. E. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. E. Hubbard, and L. D. Jackel, “Handwritten digit recognition with a back-propagation network,” in Advances in neural information processing systems, 1990, pp. 396–404.
[32] C. J. Watkins and P. Dayan, “Q-learning,” Machine learning, vol. 8, no. 3-4, pp. 279– 292, 1992.
[33] L.-J. Lin, “Reinforcement learning for robots using neural networks,” Carnegie- Mellon Univ Pittsburgh PA School of Computer Science, Tech. Rep., 1993.
[34] H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning.” in AAAI, 2016, pp. 2094–2100.
[35] E. Bjo ̈rnson, L. Sanguinetti, J. Hoydis, and M. Debbah, “Designing multi-user mimo for energy efficiency: When is massive mimo the answer?” in Wireless Communica- tions and Networking Conference (WCNC), 2014 IEEE. IEEE, 2014, pp. 242–247.
[36] G. Barlacchi, M. De Nadai, R. Larcher, A. Casella, C. Chitic, G. Torrisi, F. Antonelli, A. Vespignani, A. Pentland, and B. Lepri, “A multi-source dataset of urban life in the city of Milan and the Province of Trentino.” Scientific data, vol. 2, p. 150055, 2015.
[37] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
[38] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in neural information processing systems, 2014, pp. 3104– 3112.
[39] A.GravesandN.Jaitly,“Towardsend-to-endspeechrecognitionwithrecurrentneural networks,” in Proceedings of the 31st International Conference on Machine Learning (ICML-14), 2014, pp. 1764–1772.
[40] D.Tran,L.Bourdev,R.Fergus,L.Torresani,andM.Paluri,“Learningspatiotemporal features with 3d convolutional networks,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489–4497.
[41] S. Ji, W. Xu, M. Yang, and K. Yu, “3d convolutional neural networks for human action recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 1, pp. 221–231, 2013.
[42] J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, “Long-term recurrent convolutional networks for visual recognition and description,” in The IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), 2015.
[43] K. Y. Chan, T. S. Dillon, J. Singh, and E. Chang, “Neural-Network-Based Models for Short-Term Traffic Flow Forecasting Using a Hybrid Exponential Smoothing and Levenberg–Marquardt Algorithm,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 2, pp. 644–654, Jun. 2012.