This paper overviews one of the most important, interesting, and challenging problems in oncology, early diagnosis of prostate cancer. Developing effective diagnostic techniques for prostate cancer is of great clinical importance and can improve the effectiveness of treatment and increase the patient's chance of survival. The main focus of this study is to overview the different in-vitro and in-vivo technologies for diagnosing prostate cancer. This review discusses the current clinically used in-vitro cancer diagnostic tools, such as biomarker tests and needle biopsies and including their applications, advantages, and limitations. Moreover, the current in-vitro research tools that focus on the role of nanotechnology in prostate cancer diagnosis have been detailed. In addition to the in-vitro techniques, the current study discusses in detail developed in-vivo non-invasive state-of-the-art Computer-Aided Diagnosis (CAD) systems for prostate cancer based on analyzing Transrectal Ultrasound (TRUS) and different types of magnetic resonance imaging (MRI), e.g., T2-MRI, Diffusion Weighted Imaging (DWI), Dynamic Contrast Enhanced (DCE)-MRI, and multi-parametric MRI, focusing on their implementation, experimental procedures, and reported outcomes. Furthermore, the paper addresses the limitations of the current prostate cancer diagnostic techniques, outlines the challenges that these techniques face, and introduces the recent trends to solve these challenges, which include biomarkers used in in-vitro lab-on-a-chip nanotechnology-based methods.