In this Letter, a spherical retro-modulator architecture is introduced for operation as a bidirectional transceiver in passive optical wireless communication links. The architecture uses spherical retroreflection to enable retroreflection with broad directionality (2π steradians), and it uses all-optical beam interaction to enable modulation on ultrafast timescales (120 fs duration). The spherical retro-modulator is investigated from a theoretical standpoint and is fabricated for testing with three glasses, N-BK7, N-LASF9, and S-LAH79. It is found that the S-LAH79 structure provides the optimal refraction and nonlinearity for the desired retroreflection and modulation capabilities.