A temperature variation can electrically polarize a pyroelectric material. In its converse manifestation, the electrocaloric effect entails a change in temperature due to the application of an electric field. These phenomena have wide applications ranging from infrared detection sensors and solid-state refrigeration to energy harvesting. However, the pyroelectric-electrocaloric effect is typically observed in certain classes of hard, brittle crystalline materials that must satisfy a stringent set of lattice symmetry conditions. Some limited experiments have however demonstrated that embedding immobile charges and dipoles in soft foams (thus creating an electret state) may lead to a pyroelectric-like response as well as large deformations desired from soft matter. In this work, we develop a systematic theory for coupled electrical, thermal and mechanical responses of soft electrets. Using simple illustrative examples, we derive closed-form explicit expressions for the pyroelectric and electrocaloric coefficients of electrets. While pyroelectricity in electrets has been noted before, our derived expressions provide a clear quantitative basis to interpret (and eventually design) this effect as well as insights into how the geometrically nonlinear deformation and Maxwell stress give rise to its emergence. We present conditions to obtain a larger pyroelectric and electrocaloric response. In particular, the electrocaloric effect is predicted for the first time in such materials and we show that a proper design and a reasonable choice of materials can lead to a temperature reduction of as much as 1.5 K under the application of electrical fields of 10 MV cm-1.