Variations in tumor volume segmentation methods in targeted radionuclide therapy (TRT) may lead to dosimetric uncertainties. This work investigates the impact of PET and MRI threshold-based tumor segmentation on TRT dosimetry in patients with primary and metastatic brain tumors. In this study, PET/CT images of five brain cancer patients were acquired at 6, 24, and 48 h post-injection of 124I-CLR1404. The tumor volume was segmented using two standardized uptake value (SUV) threshold levels, two tumor-to-background ratio (TBR) threshold levels, and a T1 Gadolinium-enhanced MRI threshold. The dice similarity coefficient (DSC), jaccard similarity coefficient (JSC), and overlap volume (OV) metrics were calculated to compare differences in the MRI and PET contours. The therapeutic 131I-CLR1404 voxel-level dose distribution was calculated from the 124I-CLR1404 activity distribution using RAPID, a Geant4 Monte Carlo internal dosimetry platform. The TBR, SUV, and MRI tumor volumes ranged from 2.3-63.9 cc, 0.1-34.7 cc, and 0.4-11.8 cc, respectively. The average ± standard deviation (range) was 0.19 ± 0.13 (0.01-0.51), 0.30 ± 0.17 (0.03-0.67), and 0.75 ± 0.29 (0.05-1.00) for the JSC, DSC, and OV, respectively. The DSC and JSC values were small and the OV values were large for both the MRI-SUV and MRI-TBR combinations because the regions of PET uptake were generally larger than the MRI enhancement. Notable differences in the tumor dose volume histograms were observed for each patient. The mean (standard deviation) 131I-CLR1404 tumor doses ranged from 0.28-1.75 Gy GBq-1 (0.07-0.37 Gy GBq-1). The ratio of maximum-to-minimum mean doses for each patient ranged from 1.4-2.0. The tumor volume and the interpretation of the tumor dose is highly sensitive to the imaging modality, PET enhancement metric, and threshold level used for tumor volume segmentation. The large variations in tumor doses clearly demonstrate the need for standard protocols for multimodality tumor segmentation in TRT dosimetry.