Date of Award

January 2021

Document Type


Degree Name

Doctor of Philosophy (PhD)


Electrical Engineering

First Advisor

Saleh S. Faruque


The overall objective of this dissertation is to improve the capacity of 5G and Beyond systems by improving spectral efficiency, efficiently managing interference, and addressing the radio spectrum’s poor management while enhancing cost, energy, and computation efficiencies. The first contribution of this dissertation consists of designing effective hybrid beamforming solutions using the theory of deep reinforcement learning. The results demonstrate that deep reinforcement learning achieves near-optimal spectral efficiency and provides autonomous decision-making that can learn from interacting with the wireless environment. This method enhances hybrid beamforming computational efficiency, hardware efficiency, and energy efficiency. These methods are also desirable in scenarios where channel conditions change too fast, and we may not have existing channel datasets or the corresponding optimal beamforming solutions required for supervised learning. The second contribution of this dissertation is channel estimation in a hybrid architecture. Our numerical results demonstrated that compressive sensing could be leveraged to estimate the channel in hybrid architecture from a few training samples. The sensing matrix can be optimized, and the high dimensional channel can also be estimated using very few training samples. Our results revealed that orthogonal matching pursuit enables compressive channel estimation in low signal-to-noise ratio settings. We explored deep learning and deep reinforcement learning methods for channel estimation to design an end-to-end channel estimation where the model can map the training pilots directly to the channel estimate. The third contribution of this dissertation discusses the efficient design of jamming detection and mitigation algorithms. We developed jamming detection methods based on boosting decision trees. Our results indicated that boosting techniques achieve adequate detection performance and can be trained in a short time. The last contribution of this dissertation consists of developing narrow band and wideband spectrum sensing techniques based on machine learning and compressive sensing, respectively. We demonstrated that Bayesian compressive sensing with a Toeplitz measurement matrix could sense the activity of the primary users with high probabilities of detection while reducing the number of acquired samples, resulting in a reduction of the sensing time and complexity of the algorithm.