A major obstacle in performing multicenter controlled trials for pemphigus is the lack of a validated disease activity scoring system. Here, we assess the reliability and convergent validity of the PDAI (pemphigus disease area index). A group of 10 dermatologists scored 15 patients with pemphigus to estimate the inter- and intra-rater reliability of the PDAI and the recently described ABSIS (autoimmune bullous skin disorder intensity score) instrument. To assess convergent validity, these tools were also correlated with the Physician's Global Assessment (PGA). Reliability studies demonstrated an intra-class correlation coefficient (ICC) for inter-rater reliability of 0.76 (95% confirdence interval (CI)=0.61-0.91) for the PDAI and 0.77 (0.63-0.91) for the ABSIS. The tools differed most in reliability of assessing skin activity, with an ICC of 0.39 (0.17-0.60) for the ABSIS and 0.86 (0.76-0.95) for the PDAI. Intra-rater test-retest reliability demonstrated an ICC of 0.98 (0.96-1.0) for the PDAI and 0.80 (0.65-0.96) for the ABSIS. The PDAI also correlated more closely with the PGA. We conclude that the PDAI is more reproducible and correlates better with physician impression of extent. Subset analysis suggests that for this population of mild-to-moderate disease activity, the PDAI captures more variability in cutaneous disease than the ABSIS.