The objective of this study is to examine key challenges hindering the implementation of E-learning systems in Ghanaian tertiary institutions and to recommend practical measures that would ensure the attainment of its fullest potential. Most institutes of higher learning in Ghana have initiated reforms geared towards E-learning adoption. However, the execution of this laudable concept of teaching and learning has not been without its associated implementation challenges. Thus, affecting the realization of all the advantages associated with E-learning. The objective of the study therefore is to ascertain key problems hampering the implementation and utilization of successful E-learning systems in Ghanaian tertiary institutions. Data for the study was gathered through the deployment of questionnaire, interview and secondary data. The survey comprises of 163 respondents made up of School authorities, Faculty members, ICT staff and students of 4 tertiary institutions in Ghana. The results revealed that the introduction of E-learning comes with inherent challenges that must be dealt with by both Government and implementing tertiary institutions in order to derive all the advantages that comes with it. The study concluded by proposing conceptual recommendations that will enhance the success rate of E-learning implementation by institutes of higher learning in Ghana. In addition, the study suggests that the opportunities with this mode of teaching and learning by far surpass the challenges.