Genomic prediction based on additive genetic effects can accelerate genetic gain. There are opportunities for further improvement by including non-additive effects that access untapped sources of genetic diversity. Several studies have reported a worrying gap between the projected global future demand for plant-based products and the current annual rates of production increase, indicating that enhancing the rate of genetic gain might be critical for future food security. Therefore, new breeding technologies and strategies are required to significantly boost genetic improvement of future crop cultivars. Genomic selection (GS) has delivered considerable genetic gain in animal breeding and is becoming an essential component of many modern plant breeding programmes as well. In this paper, we review the lessons learned from implementing GS in livestock and the impact of GS on crop breeding, and discuss important features for the success of GS under different breeding scenarios. We highlight major challenges associated with GS including rapid genotyping, phenotyping, genotype-by-environment interaction and non-additivity and give examples for opportunities to overcome these issues. Finally, the potential of combining GS with other modern technologies in order to maximise the rate of crop genetic improvement is discussed, including the potential of increasing prediction accuracy by integration of crop growth models in GS frameworks.