Pharmaceuticals are ubiquitous in the natural environment with concentrations expected to rise as human population increases. Environmental risk assessments are available for a small portion of pharmaceuticals in use, raising concerns over the potential risks posed by other drugs that have little or no data. With >1900 active pharmaceutical ingredients in use, it would be a major task to test all of the compounds with little or no data. Desk-based prioritization studies provide a potential solution by identifying those substances that are likely to pose the greatest risk to the environment and which, therefore, need to be considered a priority for further study. The aim of this review was to (1) provide an overview of different prioritization exercises performed for pharmaceuticals in the environment and the results obtained; and (2) propose a new holistic risk-based prioritization framework for drugs in the environment. The suggested models to underpin this framework are discussed in terms of validity and applicability. The availability of data required to run the models was assessed and data gaps identified. The implementation of this framework may harmonize pharmaceutical prioritization efforts and ensure that, in the future, experimental resources are focused on molecules, endpoints, and environmental compartments that are biologically relevant.