Massive multiple-input multiple-output (MIMO) systems are at the core of next-generation wireless communications because of their promise of high spectral efficiency, better signal quality, and power savings. However, the optimal beamforming design for such systems is an extremely compute-intensive problem, particularly when channel conditions vary, and the channel state information (CSI) is imperfect. This paper presents a reinforcement learning approach to beamforming that bypasses these limitations through an adaptive and data-driven policy learning paradigm for efficient transmission. The Q-learning-based reinforcement learning (RL) algorithm was run for 500 episodes and showed steady convergence, with the training loss decreasing from 0.87 to 0.04 and the validation loss from 0.90 to 0.05. Prominent performance parameters delivered by this approach constituted mean signal to interference plus noise ratio (SINR) is 22.9 dB, spectral efficiency 14.8 bits/s/Hz, and BER 3.1 × 10−4 at 15 dB signal to noise ratio (SNR). The SINR obtained could be improved by 50% and 22.5% with respect to maximum ratio transmission (MRT) and zero-forcing (ZF) techniques, respectively, while the power used was 9.6 W. Robustness testing with 5% CSI error and 100 Hz Doppler fading scenarios showed the very least deterioration in performance. This leads to the conclusion that RL in beamforming is a promising approach for real-time systems in view of future adaptive scalable MIMO deployments. This work can be extended in multi-user and multi-cell settings for more general applicability.
TarafderPChoiW. Deep reinforcement learning-based coordinated beamforming for mmWave massive MIMO vehicular networks. Sensors2023; 23: 2772.
2.
AhmedIShahidMKKhammariH, et al.Machine learning based beam selection with low complexity hybrid beamforming design for 5G massive MIMO systems. IEEE Trans Green Commun Netw2021; 5: 2160–2173.
3.
GeJLiangYCZhangL, et al.Deep reinforcement learning for distributed dynamic coordinated beamforming in massive MIMO cellular networks. IEEE Trans Wireless Commun2023; 23: 4155–4169.
4.
AhmedIShahidMKFaisalT. Deep reinforcement learning based beam selection for hybrid beamforming and user grouping in massive MIMO-NOMA system. IEEE Access2022; 10: 89519–89533.
5.
AnQSegarraSDickC, et al.A deep reinforcement learning-based resource scheduler for massive MIMO networks. IEEE Trans Mach Learn Commun Netw2023; 1: 242–257.
6.
ParanthamanRNSonkerAVaralakshmiS, et al.Reinforcement learning-based model for the prevention of beam-forming vector attacks on massive MIMO system. Opt Quantum Electron2024; 56: 44.
7.
FoziMSharafatARBennisM. Fast MIMO beamforming via deep reinforcement learning for high mobility mmWave connectivity. IEEE J Sel Areas Commun2021; 40: 127–142.
8.
ArjouneYFaruqueS. Experience-driven learning-based intelligent hybrid beamforming for massive MIMO mmWave communications. Phys Commun2022; 51: 101534.
9.
AhmedAMAhmadAAFortunatiS, et al.A reinforcement learning based approach for multitarget detection in massive MIMO radar. IEEE Trans Aerosp Electron Syst2021; 57: 2622–2636.
10.
LopesVHLNahumCVDreifuerstRM, et al.Deep reinforcement learning-based scheduling for multiband massive MIMO. IEEE Access2022; 10: 125509–125525.
11.
NaeemMDe PietroGCoronatoA. Application of reinforcement learning and deep learning in multiple-input and multiple-output (MIMO) systems. Sensors2021; 22: 309.
12.
LavdasSGkonisPKZinonosZ, et al.A machine learning adaptive beamforming framework for 5G millimeter wave massive MIMO multicellular networks. IEEE Access2022; 10: 91597–91609.
13.
EappenGCosmasJNilavalanR, et al.Deep learning integrated reinforcement learning for adaptive beamforming in B5G networks. IET Commun2022; 16: 2454–2466.
14.
ChuMLiuALauVK, et al.Deep reinforcement learning based end-to-end multiuser channel prediction and beamforming. IEEE Trans Wireless Commun2022; 21: 10271–10285.
15.
ZhaiWWangXCaoX, et al.Reinforcement learning based dual-functional massive MIMO systems for multi-target detection and communications. IEEE Trans Signal Process2023; 71: 741–755.
16.
BisheFKocALe-NgocT. Deep reinforcement learning-based sum-rate maximization in hybrid beamforming multi-user massive MIMO systems. In: 2024 Tenth international conference on communications and electronics (ICCE), July 2024, pp.601–606. IEEE.
17.
ZhangYAlrabeiahMAlkhateebA. Reinforcement learning of beam codebooks in millimeter wave and terahertz MIMO systems. IEEE Trans Commun2021; 70: 904–919.
18.
WangXSunGXinY, et al.Deep transfer reinforcement learning for beamforming and resource allocation in multi-cell MISO-OFDMA systems. IEEE Trans Signal Inf Process Over Netw2022; 8: 815–829.
19.
GeJZhangLLiangYC, et al.Deep reinforcement learning for distributed coordinated beamforming in massive MIMO. In: 2023 IEEE 34th annual international symposium on personal, indoor and mobile radio communications (PIMRC), September 2023, pp.1–6. IEEE.
20.
AhmadINarmeenRBecvarZ, et al.Machine learning-based beamforming for unmanned aerial vehicles equipped with reconfigurable intelligent surfaces. IEEE Wirel Commun2022; 29: 32–38.
21.
FredjFAl-EryaniYMaghsudiS, et al.Distributed beamforming techniques for cell-free wireless networks using deep reinforcement learning. IEEE Trans Cogn Commun Netw2022; 8: 1186–1201.
22.
DangXTNguyenHVShinOS. Optimization of IRS-NOMA-assisted cell-free massive MIMO systems using deep reinforcement learning. IEEE Access2023; 11: 94402–94414.
23.
LavdasSGkonisPKTsaknakiE, et al.A deep learning framework for adaptive beamforming in massive MIMO millimeter wave 5G multicellular networks. Electronics (Basel)2023; 12: 3555.
24.
SalhAAlhartomiMAHussainGA, et al.Deep reinforcement learning-driven hybrid precoding for efficient mm-wave multi-user MIMO systems. J Sens Actuator Netw2025; 14: 20.
25.
QiaoYNiuYSuL, et al.Deep reinforcement learning based MmWave beam alignment for V2I communications. IEEE Trans Mach Learn Commun Netw2024; 2: 1216–1228.
26.
ZhuYShiELiuZ, et al.Multi-agent reinforcement learning-based joint precoding and phase shift optimization for RIS-aided cell-free massive MIMO systems. IEEE Trans Veh Technol2024; 73: 14015–14020.
27.
Al-EryaniYAkroutMHossainE. Antenna clustering for simultaneous wireless information and power transfer in a MIMO full-duplex system: a deep reinforcement learning-based design. IEEE Trans Commun2021; 69: 2331–2345.
28.
FengZClerckxB. Deep reinforcement learning for multi-user massive MIMO with channel aging. IEEE Trans Mach Learn Commun Network2023; 1: 360–375.
29.
ZhouXChenXTongL, et al.Attention-deep reinforcement learning jointly beamforming based on tensor decomposition for RIS-assisted V2X mmWave massive MIMO system. Complex & Intell Syst2024; 10: 145–160.
30.
BendjillaliRIBendelhoumMSTadjeddineAA, et al.Deep learning-powered beamforming for 5G massive MIMO systems. J Telecommun Inform Technol2023; 4: 38–45.
GaoZWuMHuC, et al.Data-driven deep learning based hybrid beamforming for aerial massive MIMO-OFDM systems with implicit CSI. IEEE J Sel Areas Commun2022; 40: 2894–2913.
33.
ElHalawanyBMHashimaSHatanoK, et al.Leveraging machine learning for millimeter wave beamforming in beyond 5G networks. IEEE Syst J2021; 16: 1739–1750.
34.
RiadhusinRSudhakarKPrakashDB, et al.Hybrid beamforming optimization in millimeter wave MIMO using deep deterministic policy gradient based deep reinforcement learning. In: 2025 3rd international conference on integrated circuits and communication systems (ICICACS), February 2025, pp.1–6. IEEE.
35.
NandhiniRDenyJ. Intelligent beamforming by deep reinforcement learning for mmWave massive MIMO by implementing NOMA for futuristic communication. In: Sustainable materials and technologies in VLSI and information processing. Boca Raton, FL: CRC Press, 2025, pp.85–91.
36.
BhuyanMSarmaKKMisraDD, et al.Adaptive hybrid beamforming codebook design using multi-agent reinforcement learning for multiuser multiple-input–multiple-output systems. Appl Sci2024; 14: 7109.
YanYZhangBLiC, et al.A novel model-assisted decentralized multi-agent reinforcement learning for joint optimization of hybrid beamforming in massive MIMO mmWave systems. IEEE Trans Veh Technol2023; 72: 14743–14755.
39.
NguyenTTNguyenKK. A deep learning framework for beam selection and power control in massive MIMO-millimeter-wave communications. IEEE Trans Mob Comput2022; 22: 4374–4387.
