Transmit Antenna Selection for Full-Duplex Relay Millimeter-Wave Communications Employing Rate-Splitting Multiple Access

This article suggests using rate-splitting multiple access (RSMA) and transmit antenna selection (TAS) within a full-duplex relay (FDR) system that operates at high millimeter-wave (MW) frequencies. We perform a mathematical analysis to derive closed-form formulas for the outage probabilities (OPs) and throughputs for two users in the proposed RSMA-MW system with TAS, taking into account the effects of residual self-interference (RSI) at the FDR. Additionally, we consider practical system and channel parameters in our analysis. The numerical results indicate that the OPs in the RSMA-MW system are significantly lower than those in a conventional nonorthogonal multiple access (NOMA)-MW system. Importantly, high MW frequencies have a considerable impact on the OPs and throughputs of the RSMA-MW system. In this context, using multiple antennas at the base station with TAS can greatly enhance the performance of the RSMA-MW system. Furthermore, we thoroughly assess the effects of RSI caused by FDR, the distances between transmitters and receivers, transmit power, common and private rates, power allocation coefficients, and frequencies. Based on the observed behaviors of the RSMA-MW system, several strategies are proposed to reduce OPs and increase throughputs. Finally, Monte Carlo simulations are performed to validate the derived formulas. © 2025 John Wiley & Sons Ltd.