While Nature harnesses RNA and DNA to store, read and write genetic information, the inherent programmability, synthetic accessibility and wide functionality of these nucleic acids make them attractive tools for use in a vast array of applications. In medicine, antisense oligonucleotides (ASOs), siRNAs, and therapeutic aptamers are explored as potent targeted treatment and diagnostic modalities, while in the technological field oligonucleotides have found use in new materials, catalysis, and data storage. The use of natural oligonucleotides limits the possible chemical functionality of resulting technologies while inherent shortcomings, such as susceptibility to nuclease degradation, provide obstacles to their application. Modified oligonucleotides, at the level of the nucleobase, sugar and/or phosphate backbone, are widely used to overcome these limitations. This review provides the reader with an overview of non-native modifications and the challenges faced in the design, synthesis, application and outlook of novel modified oligonucleotides.