We aimed to evaluate a fluorescent-labeled single chain variable fragment (scFv) of the anti-PSMA antibody as a specific probe for the detection of prostate cancer by in vivo fluorescence imaging. An orthotopic model of prostate cancer was generated by injecting LNCaP cells into the prostate lobe. ScFvD2B, a high affinity anti-PSMA antibody fragment, was labeled using a near-infrared fluorophore to generate a specific imaging probe (X770-scFvD2B). PSMA-unrelated scFv-X770 was used as a control. Probes were injected intravenously into mice with prostate tumors and fluorescence was monitored in vivo by fluorescence molecular tomography (FMT). In vitro assays showed that X770-scFvD2B specifically bound to PSMA and was internalized in PSMA-expressing LNCaP cells. After intravenous injection, X770-scFvD2B was detected in vivo by FMT in the prostate region. On excised prostates the scFv probe co-localized with the cancer cells and was found in PSMA-expressing cells. The PSMA-unrelated scFv used as a control did not label the prostate cancer cells. Our data demonstrate that scFvD2B is a high affinity contrast agent for in vivo detection of PSMA-expressing cells in the prostate. NIR-labeled scFvD2B could thus be further developed as a clinical probe for imaging-guided targeted biopsies.