Ion conducting membranes/electrolytes have been employed extensively in some important industrial and biological systems, especially in fuel cells, water electrolyzers, gas separation, sensors and biological selective ion transport, acting as one of the core components and sometimes directly determining the device performance. However, the traditional polymeric proton exchange membranes (PEMs)/anion exchange membranes (AEMs) suffer from highly toxic preparation procedures, poor thermal and chemical stabilities, and unsatisfactory ion conductivities. This has triggered researchers worldwide to explore alternative inorganic building blocks with high ion conductivities and stabilities from the new materials library, hoping to solve the above long-lasting problems. The recent burgeoning research on two-dimensional (2D) materials has unveiled exceptionally high ionic conductivities, which raises the feasibility of fabricating high-performance nanosheet-based ion conductors/membranes. In this perspective, the recent advances in measuring and understanding the exceptionally high and anisotropic H+/OH- ion conductivities of representative 2D materials, e.g. graphene oxide (GO), vermiculite and layered double hydroxide (LDH) nanosheets, are reviewed. In particular, regarding the anisotropic ionic conduction in 2D nanosheets, possible design strategies and technological innovations for fabricating macroscopic nanosheet-based ionic conductors/membranes are proposed for maximizing the high in-plane conduction, which may serve to guide future development of high-performance industrial and biological systems relying on H+/OH- conducting membranes.