Prostate cancer (PCa) is the most common type of cancer and the second leading cause of cancer-related death in men. Despite extensive research, the molecular mechanisms underlying PCa initiation and progression remain unclear, and there is increasing need of better biomarkers that can distinguish indolent from aggressive and life-threatening disease. With the advent of advanced genomic technologies in the last decade, it became apparent that the human genome encodes tens of thousands non-protein-coding RNAs (ncRNAs) with yet to be discovered function. It is clear now that the majority of ncRNAs exhibit highly specific expression patterns restricted to certain tissues and organs or developmental stages and that the expression of many ncRNAs is altered in disease and cancer, including cancer of the prostate. Such ncRNAs can serve as important biomarkers for PCa diagnosis, prognosis, or prediction of therapy response. In this review, we give an overview of the different types of ncRNAs and their function, describe ncRNAs relevant for the diagnosis and prognosis of PCa, and present emerging new aspects of ncRNA research that may contribute to the future utilization of ncRNAs as clinically useful therapeutic targets.