The self-assembly of proteins into fibrillar structures called amyloid fibrils underlies the onset and symptoms of neurodegenerative diseases, such as Alzheimer's and Parkinson's. However, the molecular basis and mechanism of amyloid aggregation are not completely understood. For many amyloidogenic proteins, certain oligomeric intermediates that form in the early aggregation phase appear to be the principal cause of cellular toxicity. Recent computational studies have suggested the importance of nonspecific interactions for the initiation of the oligomerization process prior to the structural conversion steps and template seeding, particularly at low protein concentrations. Here, using advanced single-molecule fluorescence spectroscopy and imaging of a model SH3 domain, we obtained direct evidence that nonspecific aggregates are required in a two-step nucleation mechanism of amyloid aggregation. We identified three different oligomeric types according to their sizes and compactness and performed a full mechanistic study that revealed a mandatory rate-limiting conformational conversion step. We also identified the most cytotoxic species, which may be possible targets for inhibiting and preventing amyloid aggregation.