Single molecule fluorescent lifetime trajectories of surface immobilized double-stranded DNA coupled with a tetramethylrhodmaine and Cy5 FRET pair were directly measured using time-tagged single-photon counting and scanning confocal microscopy. A modified maximum likelihood estimator (MLE) was developed to compensate for localized background fluorescence and instrument response. With this algorithm, we were able to robustly extract fluorescent lifetimes from their respective decays with as few as 20 photons. Fluorescent lifetimes extracted using an MLE were found to be highly dependent on background fluorescence. We show that appropriate factors are required to extract true lifetime trajectories from single fluorophores.