隨著衛星通訊與天地一體化網路（Space-Air-Ground Integrated Network, SAGIN）的快速發展，異質節點間的高效協作與能量最佳化仍然充滿挑戰，特別是在低軌衛星（LEO）、高空平台（HAPs）以及地面基站（BSs）組成的多層架構中，大規模部署、不均勻分布以及動態鏈路條件使得資源分配與學習穩定性變得更加複雜。為解決這些問題，本研究提出一種結合水平與垂直聯邦學習的階層式混合深度強化學習框架，用於跨層自適應資源管理。具體而言，基於深度確定性策略梯度（DDPG）的動態分群機制能依據位置、訊噪比以及傳輸條件選擇聯邦節點；同時，基於多智能體 DDPG（MADDPG）的策略則能在動態環境下聯合最佳化頻寬、功率與運算頻率，以提升能量效率。模擬結果顯示，所提方法在能量效率、收斂速度與延遲方面均優於傳統基準方法，進而能在高度動態的 SAGIN 環境中確保穩定的學習與高效的資源利用。

With the rapid development of satellite communications and Space-Air-Ground Integrated Networks (SAGIN), efficient collaboration and energy optimization across heterogeneous nodes remain challenging due to large-scale deployment, uneven distribution, and dynamic link conditions among Low Earth Orbit (LEO) satellites, High-Altitude Platforms (HAPs), and Base Stations (BSs). To address these challenges, this study proposes a hierarchical federated hybrid deep reinforcement learning framework that integrates horizontal and vertical federated learning for adaptive cross-layer resource management. Specifically, a Deep Deterministic Policy Gradient (DDPG)-based dynamic clustering mechanism selects federated nodes according to location, signal-to-noise ratio, and transmission conditions, while a Multi-Agent DDPG (MADDPG)-based strategy jointly optimizes bandwidth, power, and computing frequency to improve energy efficiency under dynamic environments. Simulation results demonstrate that the proposed method outperforms conventional baselines in terms of energy efficiency, convergence speed, and latency, thereby ensuring stable learning and efficient resource utilization in highly dynamic SAGIN scenarios.

[1] V. Bankey, S. Sharma, S. R, and A. S. Madhukumar, “Physical layer security of haps based space–air–ground-integrated network with hybrid FSO/RF communication,” IEEE Transactions on Aerospace and Electronic Systems, vol. 59, no. 4, pp. 4680 4688, August 2023.
[2] J. Zhou, S. Dang, B. Shihada, and M.-S. Alouini, “On the outage performance of space-air-ground integrated networks in the 3D poisson field,” IEEE Transactions on Vehicular Technology, vol. 73, no. 3, pp. 4401– 4406, March 2024.
[3] W. Zhu, X. Deng, J. Gui, H. Zhang, and G. Min, “Cost-effective task offloading and resource scheduling for mobile edge computing in 6G space-air–ground integrated network,” IEEE Internet of Things Journal, vol. 12, no. 12, pp. 19428– 19442, June 2025.
[4] Z. Lin, Z. Chen, Z. Fang, X. Chen, X. Wang, and Y. Gao, “FedSN: a federated learning framework over heterogeneous LEO satellite networks,” IEEE Transactions on Mobile Computing, vol. 24, no. 3, pp. 1293– 1307, March 2025.
[5] Z. Zhai, Q. Wu, S. Yu, R. Li, F. Zhang, and X. Chen, “FedLEO: an offloading-assisted decentralized federated learning framework for low earth orbit satellite networks,” IEEE Transactions on Mobile Computing, vol. 23, no. 5, pp. 5260– 5279, May 2024.
[6] M. Elmahallawy, T. Luo, and K. Ramadan, “Communication-efficient federated learning for LEO constellations integrated with HAPs using hybrid NOMA-OFDM,” IEEE Journal on Selected Areas in Communications, vol. 42, no. 5, pp. 1097– 1114, May 2024.
[7] M. Al-Hawawreh and M. S. Hossain, “Federated learning-assisted distributed intrusion detection using mesh satellite nets for autonomous vehicle protection,” IEEE Transactions on Consumer Electronics, vol. 70, no. 1, pp. 854– 862, February 2024.
[8] L. Zhao, S. Geng, X. Tang, A. Hawbani, Y. Sun, and L. Xu, “ALANINE: a novel decentralized personalized federated learning for heterogeneous LEO satellite constellation,” IEEE Transactions on Mobile Computing, vol. 24, no. 8, pp. 6945– 6960, August 2025.
[9] S. Park, S. Jung, and J. Kim, “Dynamic quantum federated learning for satellite ground integrated systems using slimmable quantum neural networks,” IEEE Access, vol. 12, pp. 58239– 58247, April 2024.
[10] Y. Formery, L. Mendiboure, J. Villain, V. Deniau, and C. Gransart, “A framework to design efficent blockchain-based decentralized federated learning architectures,” IEEE Open Journal of the Computer Society, vol. 5, pp. 705– 723, October 2024.
[11] M. R. Jabbarpour, B. Javadi, P. H. Leong, R. N. Calheiros, and D. Boland, “FedOrbit: energy efficient federated learning for orbital edge computing using block minifloatarithmetic,” IEEE Transactions on Services Computing, vol. 17, no. 6, pp. 3657 3671, Nov.-Dec. 2024.
[12] Y. Huang, X. Li, M. Zhao, H. Li, and M. Peng, “Asynchronous federated learning via over-the-air computation in LEO satellite networks,” IEEE Transactions on Wireless Communications, vol. 23, no. 12, pp. 19885– 19901, December 2024.
[13] Z. Yan and D. Li, “Convergence time optimization for decentralized federated learning with LEO satellites via number control,” IEEE Transactions on Vehicular Technology, vol. 73, no. 3, pp. 4517– 4522, March 2024.
[14] L. Chen, L. Fan, X. Lei, T. Q. Duong, A. Nallanathan, and G. K. Karagiannidis, “Relay-assisted federated edge learning: performance analysis and system optimiza tion,” IEEE Transactions on Communications, vol. 71, no. 6, pp. 3387– 3401, June 2023.
[15] N. Razmi, B. Matthiesen, A. Dekorsy, and P. Popovski, “On-board federated learning for satellite clusters with inter-satellite links,” IEEE Transactions on Communications, vol. 72, no. 6, pp. 3408– 3424, June 2024.
[16] ——, “Scheduling for on-board federated learning with satellite clusters,” in Proc. IEEE Globecom Workshops (GC Wkshps), 2023.
[17] P. Qin, D. Xu, L. Liu, M. Dong, S. Mumtaz, and M. Guizani, “Joint data allocation and LSTM-based server selection with parallelized federated learning in LEO satellite IoT networks,” IEEE Transactions on Network Science and Engineering, vol. 11, no. 6, pp. 6259– 6271, Nov.-Dec. 2024.
[18] S. Huang, L. Wang, X. Wang, B. Tan, W. Ni, and K.-K. Wong, “Edge intelligence in satellite-terrestrial networks with hybrid quantum computing,” IEEE Wireless Communications Letters, vol. 14, no. 5, pp. 1341– 1345, May 2025.
[19] L. Zou, Y. M. Park, C. M. Thwal, Y. K. Tun, Z. Han, and C. S. Hong, “Towards satellite non-IID imagery: a spectral clustering-assisted federated learning approach,” in Proc. IEEE Network Operations and Management Symposium, 2025.
[20] C.-Y. Chen, L.-H. Shen, K.-T. Feng, L.-L. Yang, and J.-M. Wu, “Edge selection and clustering for federated learning in optical inter-LEO satellite constellation,” in Proc. IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), 2023.
[21] W. Xie, C. Chen, Y. Ju, J. Shen, Q. Pei, and H. Song, “Deep reinforcement learning based computation computational offloading for space–air–ground integrated vehicle networks,” IEEE Transactions on Intelligent Transportation Systems, vol. 26, no. 5, pp. 5804– 5815, May 2025.
[22] Q. Guo, F. Tang, and N. Kato, “Routing for space-air-ground integrated network with gan-powered deep reinforcement learning,” IEEE Transactions on Cognitive Communications and Networking, vol. 11, no. 2, pp. 914– 922, April 2025.
[23] W. Pan, X. Wang, P. Zhou, and W. Lin, “Time-sensitive federated learning with heterogeneous training intensity: a deep reinforcement learning approach,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 8, no. 2, pp. 1402– 1415, April 2024.
[24] M. Jia, L. Zhang, J. Wu, Q. Guo, and X. Gu, “Asynchronous federated caching strategy for multi-satellite collaboration based on deep reinforcement learning,” IEEE Transactions on Network and Service Management, vol. 22, no. 3, pp. 2866– 2881, June 2025.
[25] H. Zhang, H. Zhao, R. Liu, X. Gao, and S. Xu, “Leader federated learning optimization using deep reinforcement learning for distributed satellite edge intelligence,” IEEE Transactions on Services Computing, vol. 17, no. 5, pp. 2544– 2557, Sept.-Oct. 2024.
[26] B. Mao, Y. Liu, Z. Wei, H. Guo, Y. Xun, and J. Wang, “A blockchain-enabled cold start aggregation scheme for federated reinforcement learning-based task offloading in zero trust LEO satellite networks,” IEEE Journal on Selected Areas in Communications, vol. 43, no. 6, pp. 2172– 2182, June 2025.
[27] N. Koursioumpas, L. Magoula, N. Petropouleas, A.-I. Thanopoulos, T. Panagea, and N. Alonistioti, “A safe deep reinforcement learning approach for energy efficient federated learning in wireless communication networks,” IEEE Transactions on Green Communications and Networking, vol. 8, no. 4, pp. 1862– 1874, December 2024.
[28] Y. Zhou, L. Lei, X. Zhao, L. You, Y. Sun, and S. Chatzinotas, “Decomposition and meta-DRL based multi-objective optimization for asynchronous federated learning in 6G-satellite systems,” IEEE Journal on Selected Areas in Communications, vol. 42, no. 5, pp. 1115– 1129, May 2024.
[29] G. Wang, F. Yang, J. Song, and Z. Han, “Dynamic laser inter-satellite link scheduling based on federated reinforcement learning: an asynchronous hierarchical architecture,” IEEE Transactions on Wireless Communications, vol. 23, no. 10, pp. 14273– 14288, October 2024.
[30] D.-J. Han, S. Hosseinalipour, D. J. Love, M. Chiang, and C. G. Brinton, “Cooperative federated learning over ground-to-satellite integrated networks: joint local computation and data offloading,” IEEE Journal on Selected Areas in Communica tions, vol. 42, no. 5, pp. 1080– 1096, May 2024.
[31] F.-H. Tseng and Y.-T. Lai, “SHFL: selective hierarchical federated learning for non-IID data distribution,” in Proc. IEEE 99th Vehicular Technology Conference (VTC2024-Spring), 2024.
[32] C. Huang, G. Chen, P. Xiao, J. A. Chambers, and W. Huang, “Fair resource allocation for hierarchical federated edge learning in space-air-ground integrated networks via deep reinforcement learning with hybrid control,” IEEE Journal on Selected Areas in Communications, vol. 42, no. 12, pp. 3618– 3631, December 2024.
[33] H.-K. Shin, K.-H. Uhm, S.-W. Jung, and S.-J. Ko, “Multispectral-to-RGB knowledge distillation for remote sensing image scene classification,” IEEE Geoscience and Remote Sensing Letters, vol. 20, February 2023.
[34] P. P. Tumpa and M. S. Islam, “Lightweight parallel convolutional neural network with svm classifier for satellite imagery classification,” IEEE Transactions on Artificial Intelligence, vol. 5, no. 11, pp. 5676– 5688, November 2024.
[35] Y. Li, S. Zhu, and J. Dai, “Joint user grouping and resource allocation for leo satellite multicast,” IEEE Systems Journal, vol. 17, no. 3, pp. 4695– 4702, September 2023.
[36] A. Talgat, M. A. Kishk, and M.-S. Alouini, “Stochastic geometry-based uplink per formance analysis of iot over leo satellite communication,” IEEE Transactions on Aerospace and Electronic Systems, vol. 60, no. 4, pp. 4198– 4213, August 2024.
[37] J. Jiao, P. Yang, Z. Du, Y. Wang, and Q. Zhang, “Clustered multi-criteria routing algorithm for mega low earth orbit satellite constellations,” IEEE Transactions on Vehicular Technology, vol. 73, no. 9, pp. 13790– 13803, September 2024.
[38] T. Yue, A. Liu, and X. Liang, “Double-layer precoder and cluster-based power allo cation design for leo satellite communication with massive mimo,” IEEE Communications Letters, vol. 27, no. 2, pp. 650– 654, February 2023.
[39] D. Wang, H. Qin, Y. Zhang, Y. Yang, and H. Lv, “Fast clustering satellite selection based on doppler positioninggdop lower bound for leo constellation,” IEEE Transactions on Aerospace and Electronic Systems, vol. 60, no. 6, pp. 9401– 9410, December 2024.
[40] S. M. A. H. Bukhari and W.-C. Song, “K-means++ clustering-based approach for sdn controller placement in leo satellite networks,” IEEE Access, vol. 13, pp. 79771– 79783, May 2025.
[41] P. E. N., M. Onyekwelu, and D. Yoon, “Extended dl coverage in sagin: Cell asso ciation and resource allocation with beam hopping leo,” IEEE Internet of Things Journal, vol. 12, no. 5, pp. 6014– 6028, March 2025.
[42] X. Qin, T. Ma, X. Zhang, Y. Wang, H. Zhou, and L. Zhao, “Ultra-dense LEO MEO constellation integrated 6G: A distributed hierarchical mobility management approach,” IEEE Transactions on Wireless Communications, vol. 24, no. 1, pp. 323– 339, January 2025.
[43] C. Liu, M. Xia, J. Zhao, H. Li, and Y. Gong, “Optimal resource allocation for integrated sensing and communications in internet of vehicles: A deep reinforcement learning approach,” IEEE Transactions on Vehicular Technology, vol. 74, no. 2, pp. 3028– 3038, February 2025.
[44] C. Liu, C. Guo, Y. Yang, and N. Jiang, “Adaptable semantic compression and re source allocation for task-oriented communications,” IEEE Transactions on Cognitive Communications and Networking, vol. 1033, no. 33, pp. 769– 782, June 2024.
[45] R. Chai, J. Liu, X. Tang, K. Gui, and Q. Chen, “A hybrid offline and online resource allocation algorithm for multibeam satellite communication systems,” IEEE Trans actions on Network and Service Management, vol. 21, no. 4, pp. 3711– 3726, August 2024.
[46] M. Nafees, S. Huang, J. Thompson, and M. Safari, “Backhaul-aware UAV-aided capacity enhancement in mixed FSO-RF network,” IEEE Open Journal of the Communications Society, vol. 5, pp. 4400– 4416, July 2024.