CaTeO3(H2O) was obtained from microwave-assisted hydrothermal synthesis as a polycrystalline sample material. The dehydration reaction was followed by thermal analysis (thermogravimetric/differential scanning calorimetry) and temperature-dependent powder X-ray diffraction and leads to a new δ-CaTeO3 polymorph. The crystal structures of CaTeO3(H2O) and δ-CaTeO3 were solved ab initio from PXRD data. CaTeO3(H2O) is non-centrosymmetric: P21cn; Z = 8; a = 14.785 49(4) Å; b = 6.791 94(3) Å; c = 8.062 62(3) Å. This layered structure is related to the ones of MTeO3(H2O) (M = Sr, Ba) with layers built of edge-sharing [CaO6(H2O)] polyhedra and are capped of each side by [Te(IV)O3E] units. Adjacent layers are stacked along the a-axis and are held together by H-bonds via the water molecules. The dehydration reaction starts above 120 °C. The transformation of CaTeO3(H2O) into δ-CaTeO3 (P21ca; Z = 8; a = 13.3647(6) Å; b = 6.5330(3) Å; c = 8.1896(3) Å) results from topotactic process with layer condensation along the a-axis and the 1/2b⃗ translation of intermediate layers. Thus, δ-CaTeO3 stays non-centrosymmetric. The characteristic layers of CaTeO3(H2O) are also maintained in δ-CaTeO3 but held together via van der Waals bonds instead of H-bonds through water molecules. Electron localization function and dipole moment calculations were also performed. For both structures and over each unit cell, the dipole moments are aligned antiparallel with net dipole moments of 3.94 and 0.47 D for CaTeO3(H2O) and δ-CaTeO3, respectively. The temperature-resolved second harmonic generation (TR-SHG) measurements, between 30 and 400 °C, show the decreasing of the SHG intensity response from 0.39 to 0.06 × quartz for CaTeO3(H2O) and δ-CaTeO3, respectively.