Magnetic nanoparticles (MNP) are intensively investigated for applications in nanomedicine, catalysis and biotechnology, where their interaction with peptides and proteins plays an important role. However, the characterisation of the interaction of individual amino acids with MNP remains challenging. Here, we classify the affinity of 20 amino acid homo-hexamers to unmodified iron oxide nanoparticles using peptide arrays in a variety of conditions as a basis to identify and rationally design selectively binding peptides. The choice of buffer system is shown to strongly influence the availability of peptide binding sites on the MNP surface. We find that under certain buffer conditions peptides of different charges can bind the MNP and that the relative strength of the interactions can be modulated by changing the buffer. We further present a model for the competition between the buffer and the MNP's electrostatically binding to the adsorption sites. Thereby, we demonstrate that the charge distribution on the surface can be used to correlate the binding of positively and negatively charged peptides to the MNP. This analysis enables us to engineer the binding of MNP on peptides and contribute to better understand the bio-nano interactions, a step towards the design of affinity tags for advanced biomaterials.