We present an embedded fragment approach for high-level quantum chemical calculations on local features in periodic systems. The fragment is defined as a set of localized orbitals (occupied and virtual) corresponding to a converged periodic Hartree-Fock solution. These orbitals serve as the basis for the in-fragment post-Hartree-Fock treatment. The embedding field for the fragment, consisting of the Coulomb and exchange potential from the rest of the crystal, is included in the fragment's one-electron Hamiltonian. As an application of the embedded fragment approach, we investigate the performance of full configuration interaction quantum Monte Carlo (FCIQMC) with the adaptive shift. As the orbital choice, we use the natural orbitals from the distinguishable cluster method with singles and doubles. FCIQMC is a stochastic approximation to the full CI method and can be routinely applied to much larger active spaces than the latter. This makes this method especially attractive in the context of open shell defects in crystals, where fragments of adequate size can be rather large. As a test case, we consider dissociation of a fluorine atom from a fluorographane surface. This process poses a challenge for high-level electronic structure models as both the static and dynamic correlations are essential here. Furthermore, the active space for an adequate fragment (32 electrons in 173 orbitals) is already quite large even for FCIQMC. Despite this, FCIQMC delivers accurate dissociation and total energies.