We present a structural and optical characterization of magnetoelastic zircon-type TmVO4 at ambient pressure and under high pressure. The properties under high pressure have been determined experimentally under hydrostatic conditions and theoretically using density functional theory. By powder X-ray diffraction we show that TmVO4 undergoes a first-order irreversible phase transition to a scheelite structure above 6 GPa. We have also determined (from powder and single-crystal X-ray diffraction) the bulk moduli of both phases and found that their compressibilities are anisotropic. The band gap of TmVO4 is found to be Eg = 3.7(2) eV. Under compression the band gap opens linearly, until it undergoes a huge collapse following the structural phase transition (ΔEg = 1.15 eV). Ab initio structural and free energy calculations support our findings. Moreover, calculations of the band structure and density of states reveal that for both zircon and scheelite TmVO4 the band gap is entirely determined by the V 3d and O 2p states of the VO43- ion. The behavior of the band gap can thus be understood entirely in terms of the structural modifications of the VO4 units under compression. Additionally, we have calculated the evolution of the infrared and Raman phonons of both phases upon compression. The presence of soft modes is related to the dynamic instability of the low-pressure phase and to the phase transition.