Since the soluble oligomers of 42-residue amyloid β (Aβ42) might be neurotoxins at an early stage of Alzheimer's disease (AD), inhibition of soluble Aβ42 oligomerization should be effective in the treatment of AD. We have found by phage display that a 7-residue peptide, SRPGLRR, exhibited inhibitory activity against soluble 37/48 kDa oligomers of Aβ42. In the present study, we newly prepared 3- and 4-residue random peptides libraries and performed pannings of them against soluble Aβ42 to search for important factors in the inhibition of Aβ42 oligomerization. After the fifth round, arginine-containing peptides were enriched in both libraries. SDS-PAGE and size-exclusion chromatography indicated that the inhibitory activities of 4-residue peptides against the soluble 37/48 kDa oligomers of Aβ42 were higher than those of the 3-residue peptides, and RFRK exhibited strong inhibitory activity as well as SRPGLRR. These short peptides should be useful for the suppression of soluble Aβ42 oligomer formation.