Alzheimer's disease (AD) is characterized by an imbalance between production and clearance of amyloid-β (Aβ) species. Aβ peptides can transform structurally from monomers into β-stranded fibrils via multiple oligomeric states. Among the various Aβ species, structured oligomers are proposed to be more toxic than fibrils; however, the identification of Aβ oligomers has been challenging due to their heterogeneous and metastable nature. Multiple techniques have recently helped us gain a better understanding of oligomers' assembly details and structural properties. Moreover, some progress on elucidating the mechanisms of oligomer-triggered toxicity has been made. Based on the collection of current findings, there is growing consensus that control of toxic Aβ oligomers could be a valid approach to regulate Aβ-associated toxicity, which could advance development of new diagnostics and therapeutics for amyloid-related diseases. In this review, we summarize the recent understanding of Aβ oligomers' assembly, structural properties, and toxicity, along with inhibitors against Aβ aggregation, including oligomerization.