Coalescence and sampling distributions for Feller diffusions

Theor Popul Biol. 2024 Feb:155:67-76. doi: 10.1016/j.tpb.2023.12.001. Epub 2023 Dec 12.

Abstract

Consider the diffusion process defined by the forward equation ut(t,x)=12{xu(t,x)}xx-α{xu(t,x)}x for t,x≥0 and -∞<α<∞, with an initial condition u(0,x)=δ(x-x0). This equation was introduced and solved by Feller to model the growth of a population of independently reproducing individuals. We explore important coalescent processes related to Feller's solution. For any α and x0>0 we calculate the distribution of the random variable An(s;t), defined as the finite number of ancestors at a time s in the past of a sample of size n taken from the infinite population of a Feller diffusion at a time t since its initiation. In a subcritical diffusion we find the distribution of population and sample coalescent trees from time t back, conditional on non-extinction as t→∞. In a supercritical diffusion we construct a coalescent tree which has a single founder and derive the distribution of coalescent times.

Keywords: Branching process; Coalescent; Diffusion process; Feller diffusion; Sampling distributions.