We present rigorous analysis of optical activity of chiral semiconductor gammadions whose chirality in three dimensions is caused by the nonuniformity of thickness in the transverse plane. It is shown that such gammadions not only distinguish between the two circular polarizations upon scattering and reflection of light, like all two-dimensional semiconductor nanostructures with planar chirality do, but also exhibit circular dichroism and circularly polarized luminescence. Chiral semiconductor gammadions whose charge carriers are mostly confined to the arms are found to feature both high dissymmetry of optical response and a constant-sign circular dichroism signal over a wide frequency range. It is also shown that the strength of the gammadion's chiroptical response is determined solely by two geometric factors: the variation range of the gammadion's thickness and the arms' curvature. Our seminal theoretical study is intended to lay the foundation for future applications of semiconductor gammadions in chiral nanophotonics and nanotechnology.