Outage analysis of MIMO-NOMA relay system with user clustering and beamforming under imperfect CSI in Nakagami-m fading channels

In this paper, we propose and analyze a downlink multiple-user MIMO-NOMA relay system where all users are grouped into several clusters. To mitigate the inter-cluster interference, the superposition signals at the base station are beamed at the relay nodes. Then, the relays communicate with the users through superposition coding in power domain. We derive the exact closed-form expressions of the outage probability of each user in every cluster for the cases of perfect successive-interference cancellation (SIC) and imperfect SIC. The outage performance is analyzed the over Nakagami-m fading channels, taking into account the imperfection of channel state information (CSI). All analysis results are compared with Monte-Carlo simulation result to verify the correctness of the derived mathematical expressions. The results show that the channel fading severity, channel estimation error, and the quality of SIC structure have strong influences on the outage performance of the proposed MIMO-NOMA relay system. © 2019, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.