Blindly increasing the channels of the mode (de)multiplexer on the single-layer chip can cause the device structure to be too complex to optimize. The three-dimensional (3D) mode division multiplexing (MDM) technology is a potential solution to extend the data capacity of the photonic integrated circuit by assembling the simple devices in the 3D space. In our work, we propose a 16 × 16 3D MDM system with a compact footprint of about 100 µm × 5.0 µm × 3.7 µm. It can realize 256 mode routes by converting the fundamental transverse electric (TE0) modes in arbitrary input waveguides into the expected modes in arbitrary output waveguides. To illustrate its mode-routing principle, the TE0 mode is launched in one of the sixteen input waveguides, and converted into corresponding modes in four output waveguides. The simulated results indicate that the ILs and CTs of the 16 × 16 3D MDM system are less than 3.5 dB and lower than -14.2 dB at 1550 nm, respectively. In principle, the 3D design architecture can be scaled to realize arbitrary network complexity levels.