The observation of a high-mobility two-dimensional electron gas between two insulating complex oxides, especially LaAlO₃/SrTiO₃, has enhanced the potential of oxides for electronics. The occurrence of this conductivity is believed to be driven by polarization discontinuity, leading to an electronic reconstruction. In this scenario, the crystal orientation has an important role and no conductivity would be expected, for example, for the interface between LaAlO₃ and (110)-oriented SrTiO₃, which should not have a polarization discontinuity. Here we report the observation of unexpected conductivity at the LaAlO₃/SrTiO₃ interface prepared on (110)-oriented SrTiO₃, with a LaAlO₃-layer thickness-dependent metal-insulator transition. Density functional theory calculation reveals that electronic reconstruction, and thus conductivity, is still possible at this (110) interface by considering the energetically favourable (110) interface structure, that is, buckled TiO₂/LaO, in which the polarization discontinuity is still present. The conductivity was further found to be strongly anisotropic along the different crystallographic directions with potential for anisotropic superconductivity and magnetism, leading to possible new physics and applications.