Formic acid (or formate) is suggested to be one of the most economically viable products from electrochemical carbon dioxide reduction. However, its commercial viability hinges on the development of highly active and selective electrocatalysts. Here we report that structural defects have a profound positive impact on the electrocatalytic performance of bismuth. Bismuth oxide double-walled nanotubes with fragmented surface are prepared as a template, and are cathodically converted to defective bismuth nanotubes. This converted electrocatalyst enables carbon dioxide reduction to formate with excellent activity, selectivity and stability. Most significantly, its current density reaches ~288 mA cm-2 at -0.61 V versus reversible hydrogen electrode within a flow cell reactor under ambient conditions. Using density functional theory calculations, the excellent activity and selectivity are rationalized as the outcome of abundant defective bismuth sites that stabilize the *OCHO intermediate. Furthermore, this electrocatalyst is coupled with silicon photocathodes and achieves high-performance photoelectrochemical carbon dioxide reduction.