The identification of fentanyl, a main culprit in opioid overdose deaths, has become critical. Whereas Raman spectroscopy is an effective tool for detecting illicit drugs, the weak intensity of Raman scattering can make it difficult to distinguish trace materials. This shortcoming is addressed by surface-enhanced Raman spectroscopy (SERS), which produces strong signal enhancements when target compounds are near metal nanoparticles. This work examines the use of a paper-based substrate impregnated with silver nanoparticles for the detection of trace quantities of fentanyl alone and as an adulterant in heroin. In addition, intensity ratios of diagnostic peaks associated with each substance were fitted to a Langmuir isotherm calibration model and used for the quantitative analysis of fentanyl in heroin mixtures. Linearity was observed at <6% fentanyl, a significant finding that is consistent with concentrations found in drugs seized during law enforcement efforts. In addition, swabbing with these paper-based SERS substrates facilitated the recovery of fentanyl from surfaces, showing this to be applicable for crime scene investigations. However, assessment using the calibration model proved difficult for swabbed samples. Overall, this work demonstrates a potentially simple and sensitive technique for the forensic analysis and quantitation of fentanyl in trace amounts.