Accurate heritability estimates for fitness-related traits are required to predict an organism's ability to respond to global change. Heritability estimates are theoretically expected to be inflated if, due to limited dispersal, individuals that share genes are also likely to share similar environments. However, if relatives occupy similar environments due, at least partly, to genetic variation for habitat selection, then accounting for environmental similarity in quantitative genetic models may result in diminished heritability estimates in wild populations. This potential issue has been pointed out in the literature, but has not been evaluated by empirical studies. Here, we investigate whether environmental similarity among individuals can be partly explained by genetic variation for habitat selection, and how this link potentially blurs estimates for heritability in fitness-related traits. Using intensive GPS monitoring, we quantified home-range habitat composition for 293 roe deer inhabiting a heterogeneous landscape to assess environmental similarity. To investigate if environmental similarity might harbour genetic variation, we combined genome-wide data in a quantitative genetic framework to evaluate genetic variation for home-range habitat composition, which is partly the result of habitat selection at settlement. Finally, we explored how environmental similarity affects heritability estimates for behaviours related to the risk avoidance-resource acquisition trade-off (i.e. being in open habitat and distance to roads) and proxies of individual performance (i.e. body mass and hind foot length). We found substantial heritability for home-range habitat composition, with estimates ranging from 0.40 (proportion of meadows) to 0.85 (proportion of refuge habitat). Accounting for similarity in habitat composition between relatives decreased the heritability estimates for both behavioural and morphological traits (reduction ranging from 55% to 100% and from 22% to 41% respectively). As a consequence, only half of these heritability estimates remained significantly different from zero. Our results show that similar genotypes occupy similar environments, which could lead to heritable variation being incorrectly attributed to environmental effects. To accurately distinguish the sources of phenotypic variation and predict the ability of organisms to respond to global change, it is necessary to develop quantitative genetic studies investigating the mechanisms underpinning environmental similarity among relatives.
Keywords: environmental similarity; genomic relatedness matrix; genotype-environment correlation; habitat selection; heritability.
© 2022 British Ecological Society.