The distribution of parasites in hosts is typically aggregated: a few hosts harbour many parasites, while the remainder of hosts are virtually parasite free. The origin of this almost universal pattern is central to our understanding of host-parasite interactions; it affects many facets of their ecology and evolution. Despite this, the standard statistical framework used to characterize parasite aggregation does not describe the processes generating such a pattern. In this work, we have developed a mathematical framework for the distribution of parasites in hosts, starting from a simple statistical description in terms of two fundamental processes: the exposure of hosts to parasites and the infection success of parasites. This description allows the level of aggregation of parasites in hosts to be related to the random variation in these two processes and to true host heterogeneity. We show that random variation can generate an aggregated distribution and that the common view, that encounters and success are two equivalent filters, applies to the average parasite burden under neutral assumptions but it does not apply to the variance of the parasite burden, and it is not true when heterogeneity between hosts is incorporated in the model. We find that aggregation decreases linearly with the number of encounters, but it depends non-linearly on parasite success. We also find additional terms in the variance of the parasite burden which contribute to the actual level of aggregation in specific biological systems. We have derived the formal expressions of these contributions, and these provide new opportunities to analyse empirical data and tackle the complexity of the origin of aggregation in various host-parasite associations.