In this paper we report a theoretical study of the CaSeO4 compound at ambient pressure and under pressure. Here we made a structural analysis of its three known polymorphs--orthorhombic (Cmca), monoclinic monazite, and tetragonal scheelite--where direct comparison with experimental measurements is done. Besides, the electronic and vibrational structures are reported for the first time for those structures. In addition, the behavior of CaSeO4 as a function of pressure is studied, where phase transitions are investigated by considering a quasiharmonic approximation at 300 K. After a total energy study of 14 possible high-pressure phases of CaSeO4, the following sequence of pressure-driven structural transitions has been found: orthorhombic (Cmca) → tetragonal scheelite → monoclinic AgMnO4-type structure. It was observed that monazite is less stable as temperature increases, while the opposite occurs for the AgMnO4-type structure, this being a novel polymorph. This high-pressure structure is a distortion of the monazite structure and resembles the distorted barite-type structure (P2(1)/n) of CaSO4. The equation of state and the pressure evolution of the structural, electronic, and vibrational properties are also reported.