The structural complexity of the antiferromagnetic oxide selenide CaFeSeO is described. The compound contains puckered FeSeO layers composed of FeSe2O2 tetrahedra sharing all their vertexes. Two polymorphs coexist that can be derived from an archetype BaZnSO structure by cooperative tilting of the FeSe2O2 tetrahedra. The polymorphs differ in the relative arrangement of the puckered layers of vertex-linked FeSe2O2 tetrahedra. In a noncentrosymmetric Cmc21 polymorph (a = 3.89684(2) Å, b = 13.22054(8) Å, c = 5.93625(2) Å) the layers are related by the C-centering translation, while in a centrosymmetric Pmcn polymorph, with a similar cell metric (a = 3.89557(6) Å, b = 13.2237(6) Å, c = 5.9363(3) Å), the layers are related by inversion. The compound shows long-range antiferromagnetic order below a Neél temperature of 159(1) K with both polymorphs showing antiferromagnetic coupling via Fe-O-Fe linkages and ferromagnetic coupling via Fe-Se-Fe linkages within the FeSeO layers. The magnetic susceptibility also shows evidence for weak ferromagnetism which is modeled in the refinements of the magnetic structure as arising from an uncompensated spin canting in the noncentrosymmetric polymorph. There is also a spin glass component to the magnetism which likely arises from the disordered regions of the structure evident in the transmission electron microscopy.