Studying the response of materials to strain can elucidate subtle properties of the electronic structure in strongly correlated materials. Here, we focus on the elastocaloric coefficients, forming a second rank tensor quantity describing the relation between entropy and strain. In contrast to the better-known elastoresistivity, the elastocaloric effect is a thermodynamic quantity. Experimentally, elastocaloric effect measurements are demanding since the thermodynamic conditions during the measurement have to be well controlled. In this work, we present a technique to measure the elastocaloric effect under quasiadiabatic conditions. The technique is based on oscillating strain, which allows for increasing the frequency of the elastocaloric effect above the thermal relaxation rate of the sample. We apply the technique to Co-doped iron pnictide superconductors and show that the thermodynamic signatures of second order phase transitions in the elastocaloric effect closely follow those observed in calorimetry experiments. In contrast to heat capacity, elastocaloric effect measurements allow for the electronic signatures to be measured against a small phononic background even at high temperatures and in addition give information on the symmetry of the involved order parameters. This establishes the technique as a powerful complimentary tool for extracting the entropy landscape as a function of strain proximate to a continuous phase transition.