Antimicrobial peptides (AMPs) have been considered as potential therapeutic sources of future antibiotics because of their broad-spectrum activities and different mechanisms of action compared to conventional antibiotics. Although AMPs possess considerable benefits as new generation antibiotics, their clinical and commercial development still have some limitations, such as potential toxicity, susceptibility to proteases, and high cost of peptide production. In order to overcome those obstacles, extensive efforts have been carried out. For instance, unusual amino acids or peptido-mimetics are introduced to avoid the proteolytic degradation and the design of short peptides retaining antimicrobial activities is proposed as a solution for the cost issue. In this review, we focus on small peptides, especially those with less than twelve amino acids, and provide an overview of the relationships between their three-dimensional structures and antimicrobial activities. The efforts to develop highly active AMPs with shorter sequences are also described.