Resistive random access memory (RRAM) is an important candidate for both digital, high-density data storage and for analog, neuromorphic computing. RRAM operation relies on the formation and rupture of nanoscale conductive filaments that carry enormous current densities and whose behavior lies at the heart of this technology. Here, we directly measure the temperature of these filaments in realistic RRAM with nanoscale resolution using scanning thermal microscopy. We use both conventional metal and ultrathin graphene electrodes, which enable the most thermally intimate measurement to date. Filaments can reach 1300°C during steady-state operation, but electrode temperatures seldom exceed 350°C because of thermal interface resistance. These results reveal the importance of thermal engineering for nanoscale RRAM toward ultradense data storage or neuromorphic operation.