Increasing agricultural productivity is one of the most important goals of plant science research and imperative to meet the needs of a rapidly growing population. Rice (Oryza sativa L.) is one of the most important staple crops worldwide. Grain size is both a major determinant of grain yield in rice and a target trait for domestication and artificial breeding. Here, a genome-wide association study of grain length and grain width was performed using 996,722 SNP markers in 270 rice accessions. Five and four quantitative trait loci were identified for grain length and grain width, respectively. In particular, the novel grain size gene OsSNB was identified from qGW7, and further results showed that OsSNB negatively regulated grain size. Most notably, knockout mutant plants by CRISPR/Cas9 technology showed increased grain length, width, and weight, while overexpression of OsSNB yielded the opposite. Sequencing of this gene from the promoter to the 3'-untranslated region in 168 rice accessions from a wide geographic range identified eight haplotypes. Furthermore, Hap 3 has the highest grain width discovered in japonica subspecies. Compared to other haplotypes, Hap 3 has a 225 bp insertion in the promoter. Based on the difference between Hap 3 and other haplotypes, OsSNB_Indel2 was designed as a functional marker for the improvement of rice grain width. This could be directly used to assist selection toward an improvement of grain width. These findings suggest OsSNB as useful for further improvements in yield characteristics in most cultivars.