車載網路中的車輛可以根據環境安排自己的傳輸模式、功率和子頻道，來盡可能地提高系統的整體效用。在過去的研究中，衛星已被用於使車輛在沒有V2V或V2I連接，或者連接品質較差時能夠有不同的選擇。但是，城市車輛往往在V2V和V2I通道品質較差的情況下選擇V2S，這將導致沒有V2X和V2I使用的農村車輛失去唯一的連接選擇。
因此，我們讓車輛在選擇通道時考慮到對其他車輛的影響。通過減少對他人的負面影響的過程，可以更合理地分配通道。使系統可用於更多車輛，並提高整體系統利用率。
在我們的模擬中，包括城市、郊區和農村地區的數據是由城市交通模擬 (SUMO) 生成的。模擬結果證實，該方法比以前的方案具有更好的性能。

Vehicles in vehicular network can adjust their transmission modes, sub-channel and transmit power according to the environment to maximize the system utility. In previous works, satellites have been used to enable vehicles to have different options when there is no V2V or V2I connection, or when the connection quality is poor. However, vehicles in urban areas often choose V2S when the quality of the V2V and V2I channels is poor, which will cause vehicles in rural areas without V2X and V2I to use lose the only alternative connection.
Therefore, we make vehicles take into account the impact on other vehicles when selecting a channel. Through the process of reducing the negative impact on others, the channels can be allocated in a more reasonable way. Make system available to more vehicles and increase overall system utilization.\\
In the simulation, we use Simulation of Urban Mobility (SUMO) to generate the data which including the urban, suburban and rural area. And the result shows that the success rate of agents in each area is more even than before.

[1] F. Lisi, G. Losquadro, A. Tortorelli, A. Ornatelli, and M. Donsante. Multi-
connectivity in 5g terrestrial-satellite networks: the 5g-allstar solution, 2020.
[2] Subramanya Chandrashekar, Andreas Maeder, Cinzia Sartori, Thomas H ̈ohne,
Benny Vejlgaard, and Devaki Chandramouli. 5g multi-rat multi-connectivity ar-
chitecture. In 2016 IEEE International Conference on Communications Workshops
(ICC), pages 180–186, 2016.
[3] Hao Ye, Geoffrey Ye Li, and Biing-Hwang Fred Juang. Deep reinforcement learning
based resource allocation for v2v communications. IEEE Transactions on Vehicular
Technology, 68(4):3163–3173, 2019.
[4] Liang Wang, Hao Ye, Le Liang, and Geoffrey Ye Li. Learn to compress csi and
allocate resources in vehicular networks. IEEE Transactions on Communications,
68(6):3640–3653, 2020.
[5] Hao Ye and Geoffrey Ye Li. Deep reinforcement learning based distributed resource
allocation for v2v broadcasting. In 2018 14th International Wireless Communica-
tions Mobile Computing Conference (IWCMC), pages 440–445, 2018.
[6] Le Liang, Hao Ye, and Geoffrey Ye Li. Spectrum sharing in vehicular networks
based on multi-agent reinforcement learning. IEEE Journal on Selected Areas in
Communications, 37(10):2282–2292, 2019.
[7] Min Zhao, Yifei Wei, Mei Song, and Guo Da. Power control for d2d communi-
cation using multi-agent reinforcement learning. In 2018 IEEE/CIC International
Conference on Communications in China (ICCC), pages 563–567, 2018.
[8] Dohyun Kwon and Joongheon Kim. Multi-agent deep reinforcement learning for
cooperative connected vehicles. In 2019 IEEE Global Communications Conference
(GLOBECOM), pages 1–6, 2019.
21
[9] Yen-Yang Wu. Intelligent multi-connectivity management for satellite-aided vehic-
ular networks. Master’s thesis, National Central University, 10 2021.
[10] Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony
Bharath. A brief survey of deep reinforcement learning. CoRR, abs/1708.05866,
2017.
[11] John N. Tsitsiklis Vijay R. Konda. Actor-critic algorithms. Advances in neural
information processing systems, page 1008–1014, 2000.
[12] Ishan Budhiraja, Neeraj Kumar, and Sudhanshu Tyagi. Deep-reinforcement-
learning-based proportional fair scheduling control scheme for underlay d2d com-
munication. IEEE Internet of Things Journal, 8(5):3143–3156, 2021.
[13] Muhammad Sohaib, Jongjin Jeong, and Sang-Woon Jeon. Dynamic multichannel
access via multi-agent reinforcement learning: Throughput and fairness guarantees.
IEEE Transactions on Wireless Communications, 21(6):3994–4008, 2022.
[14] Matthieu Zimmer, Claire Glanois, Umer Siddique, and Paul Weng. Learning fair
policies in decentralized cooperative multi-agent reinforcement learning. In Marina
Meila and Tong Zhang, editors, Proceedings of the 38th International Conference
on Machine Learning, volume 139 of Proceedings of Machine Learning Research,
pages 12967–12978. PMLR, 18–24 Jul 2021.
[15] 3GPP TR 38.821. Technical Specification Group Radio Access Network; Solutions
for NR to support non-terrestrial networks (NTN), December 2019.
[16] C. Kourogiorgas, D. Tarchi, A. Ugolini, P. D. Arapoglou, A. D. Panagopoulos,
G. Colavolpe, and A. Vanelli Coralli. System capacity evaluation of dvb-s2x based
medium earth orbit satellite network operating at ka band. In 2016 8th Advanced
Satellite Multimedia Systems Conference and the 14th Signal Processing for Space
Communications Workshop (ASMS/SPSC), pages 1–8, 2016.