Caveolin-1 (CAV1) is an important regulator of adipose tissue homeostasis. In the present study we examined the impact of CAV1 deficiency on the properties of mouse adipose tissue both in vivo and in explant cultures during conditions of metabolic stress. In CAV1(-/-) mice fasting caused loss of adipose tissue mass despite a lack of hormone-sensitive lipase (HSL) phosphorylation. In addition, fasting resulted in increased macrophage infiltration, enhanced deposition of collagen, and a reduction in the level of the lipid droplet protein perilipin A (PLIN1a). Explant cultures of CAV1(-/-) adipose tissue also showed a loss of PLIN1a during culture, enhanced secretion of IL-6, increased release of lactate dehydrogenase, and demonstrated increased susceptibility to cell death upon collagenase treatment. Attenuated PKA-mediated signaling to HSL, loss of PLIN1a and increased secretion of IL-6 were also observed in adipose tissue explants of CAV1(+/+) mice with diet-induced obesity. Together these results suggest that while alterations in adipocyte lipid droplet biology support adipose tissue metabolism in the absence of PKA-mediated pro-lipolytic signaling in CAV1(-/-) mice, the tissue is intrinsically unstable resulting in increased susceptibility to cell death, which we suggest underlies the development of fibrosis and inflammation during periods of metabolic stress.