If they undergo new mutations at each replication cycle, why are RNA viral genomes so fragile, with most mutations being either strongly deleterious or lethal? Here we provide theoretical and numerical evidence for the hypothesis that genetic fragility is partly an evolutionary response to the multiple population bottlenecks experienced by viral populations at various stages of their life cycles. Modelling within-host viral populations as multi-type branching processes, we show that mutational fragility lowers the rate at which Muller's ratchet clicks and increases the survival probability through multiple bottlenecks. In the context of a susceptible-exposed-infectious-recovered epidemiological model, we find that the attack rate of fragile viral strains can exceed that of more robust strains, particularly at low infectivities and high mutation rates. Our findings highlight the importance of demographic events such as transmission bottlenecks in shaping the genetic architecture of viral pathogens.