With interest in charge transfer compounds growing steadily, it is important to understand all aspects of the underlying physics of these systems, including the properties of the defects and interfaces that are universally present in actual experimental systems. For the study of these defects and their interactions a spin-Peierls (SP) system provides a useful testing ground. This work presents an investigation within the SP phase of potassium TCNQF4 where anomalous features are observed in both the magnetic susceptibility and ESR spectra for temperatures between 60 K and 100 K. Muon spin spectroscopy measurements confirm the presence of these anomalous magnetic features, with low temperature zero-field data exhibiting the damped oscillatory form that is a characteristic signature of static magnetic order. This ordering is most likely due to the interaction between structurally correlated magnetic defects in the system. The critical behaviour of the temperature dependent muon spin rotation frequency indicates that a 2D Ising model is applicable to the magnetic ordering of these defects. We show that these observations can be explained by a simple model in which the magnetic defects are located at stacking faults, which provide them with a 2D structural framework to constrain their interactions.