We report the thermally detected electron spin resonance (ESR) spectroscopy in the frequency range of millimeter and submillimeter waves. Under high vacuum conditions, a cantilever-shaped device detects ESR absorption of a mounted sample as a temperature difference in its beam direction. Despite the simple experimental setup, the spin sensitivity of the order of 1012 spins/G was achieved at 10 K. The developed sample stage is small enough to be used in a 10 T split-pair superconducting magnet with a bore of 25 mm, enabling precise field-angle-dependent ESR measurements at multi-frequencies above 500 GHz. We demonstrate its usefulness by studying the field-angle dependence of the excitation energy of the dimer triplet state in the Shastry-Sutherland magnet SrCu2(BO3)2.