Numerous attempts have been made to realize crossed coupling between ferroelectricity and magnetism in multiferroic materials at room temperature. BiFeO3 is the most extensively studied multiferroic material that shows multiferroicity at temperatures significantly above room temperature. Here we present high-field experiments on high-quality mono-domain BiFeO3 crystals reveal substantial electric polarization orthogonal to the widely recognized one along the trigonal c axis. This novel polarization appears to couple with the domains of the cycloidal spin order and, hence, can be controlled using magnetic fields. The transverse polarization shows the non-volatile memory effect at least up to 300 K.