Two-dimensional materials, including graphene, transition metal dichalcogenides and their heterostructures, exhibit great potential for a variety of applications, such as transistors, spintronics, and photovoltaics. While the miniaturization offers remarkable improvements in electrical performance, heat dissipation and thermal mismatch can be a problem in designing electronic devices based on two-dimensional materials. Quantifying the thermal expansion coefficient of 2D materials requires temperature measurements at nanometer scale. Here, we introduce a novel nanometer-scale thermometry approach to measure temperature and quantify the thermal expansion coefficients in 2D materials based on scanning transmission electron microscopy combined with electron energy-loss spectroscopy to determine the energy shift of the plasmon resonance peak of 2D materials as a function of sample temperature. By combining these measurements with first-principles modeling, the thermal expansion coefficients (TECs) of single-layer and freestanding graphene and bulk, as well as monolayer MoS_{2}, MoSe_{2}, WS_{2}, or WSe_{2}, are directly determined and mapped.