Previous works indicate that intensity-modulated radiotherapy (IMRT) and volumetric modulated arc therapy (VMAT) plans that are highly complex may produce more errors in dose calculation and treatment delivery. Multiple complexity metrics have been proposed and associated with IMRT QA results, but their relationships with plan performance using in situ dose measurements have not been thoroughly investigated. This study aimed to evaluate the relationships between IMRT treatment plan complexity and anthropomorphic phantom performance in order to assess the extent to which plan complexity is related to dosimetric performance in the IROC phantom credentialing program. Sixteen complexity metrics, including the modulation complexity score (MCS), several modulation indices, and total monitor units (MU) delivered, were evaluated for 343 head and neck phantom irradiations, comprising both IMRT (step-and-shoot and sliding window techniques) and VMAT. Spearman's correlations were used to explore the relationship between complexity and plan performance, as measured by the dosimetric differences between the treatment planning system (TPS) and thermoluminescent dosimeter (TLD) measurement, as well as film gamma analysis. Relationships were likewise determined for several combinations of subpopulations, based on the linear accelerator model, TPS used, and delivery modality. Evaluation of the complexity metrics presented here yielded no significant relationships (p > 0.01, Bonferroni-corrected) and all correlations were weak (less than ±0.30). These results indicate that complexity metrics have limited predictive utility in assessing plan performance in multi-institutional comparisons of IMRT plans. Other factors affecting plan accuracy, such as dosimetric modeling or multileaf collimator (MLC) performance, should be investigated to determine a more probable cause for dose delivery errors.