Zika virus and dengue virus are co-circulating flaviviruses with a widespread endemic range. Eliciting broad and potent neutralizing antibodies is an attractive goal for developing a vaccine to simultaneously protect against these viruses. However, the capacity of viral mutations to confer escape from broadly neutralizing antibodies remains undescribed, due in part to limited throughput and scope of traditional approaches. Here, we use deep mutational scanning to map how all possible single amino acid mutations in Zika virus envelope protein affect neutralization by antibodies of varying breadth and potency. While all antibodies selected viral escape mutations, the mutations selected by broadly neutralizing antibodies conferred less escape relative to those selected by narrow, virus-specific antibodies. Surprisingly, even for broadly neutralizing antibodies with similar binding footprints, different single mutations led to escape, indicating distinct functional requirements for neutralization not captured by existing structures. Additionally, the antigenic effects of mutations selected by broadly neutralizing antibodies were conserved across divergent, albeit related, flaviviruses. Our approach identifies residues critical for antibody neutralization, thus comprehensively defining the as-yet-unknown functional epitopes of antibodies with clinical potential.