A stacked integration technique based on polarization optics is studied for implementing shuffle-based interconnection networks with three-dimensional solid-state modules. A basic building block of the proposed scheme consists of a hole-patterned half-wave retarder for birefringence customizations and a calcite slab for subsequent beam deflections. On the basis of a cascade of such building blocks the submodules of various shuffle-family permutations can be implemented. To minimize channel cross talk, we incorporated a collimating-relaying imaging system. To help design birefringence customization, we developed algebraic formulations of folded shuffle operations using separable shuffles. Proof-of-concept experimental results, as well as system design, fabrication, and integration issues, are discussed.