We investigate the nature and role of krypton embedding in small neutral gold clusters. For some of these clusters, we observe a particular site-dependent character of the Kr binding that does not completely follow the criterion of binding at low-coordinated sites, widely accepted for interaction of a noble gas with closed-shell metal systems such as metal surfaces. We aim at understanding the effect of low dimensionality and open-shell electronic structure of the odd-numbered clusters on the noble gas-metal cluster interaction. First, we investigate the role of attractive and repulsive forces, and the frontier molecular orbitals. Second, we investigate the Au-Kr interaction in terms of reactivity and bonding character. We use a reactivity index derived from Fukui formalism, and criteria provided by the electron localization function (ELF), in order to classify the type of bonding. We carry out this study on the minimum energy structures of neutral gold clusters, as obtained using pseudo potential plane-wave density functional theory (DFT). A model is proposed that includes the effect of attractive electrostatic, van der Waals and repulsive forces, together with effects originating from orbital overlap. This satisfactorily explains minimum configurations of the noble gas-gold cluster systems, the site preference of the noble gas atoms, and changes in electronic properties.