A recent study by Tawfik et al. [ Phys. Rev. Mater. 2018, 2, 034005] found that few density functionals, none of which are asymptotic pairwise dispersion methods, describe the geometry and binding of layered materials accurately. Here, we show that the exchange-hole dipole moment (XDM) dispersion model attains excellent results for graphite, hexagonal BN, and transition-metal dichalcogenides. Contrary to what has been argued, successful modeling of layered materials does not necessitate meta-GGA exchange, nonlocal correlation functionals, or the inclusion of three-body dispersion terms. Rather, a GGA functional, combined with a simple asymptotic pairwise dispersion correction, can be reliably used, provided that it properly accounts for the geometric dependence of the dispersion coefficients. The overwhelming contribution to the variation of the pairwise dispersion coefficients comes from the immediate vicinity of an atom and is already present for single layers. Longer-range and interlayer effects are examined in detail for graphite.