A 71-year-old man was admitted with malaise, mild fever, anorexia, body weight loss, lower back pain, thirst, and polydipsia. He showed bilateral swelling of the submandibular glands. Examinations showed panhypopituitarism and a high serum IgG4 concentration. Fluorodeoxyglucose positron emission tomography (FDG-PET) revealed uptake in the pituitary gland, bilateral submandibular gland, bilateral hilar and mediastinal lymph nodes, and a mass consistent with retroperitoneal fibrosis, but not in the pancreas. Biopsy specimens from the submandibular gland and retroperitoneal mass indicated sialadenitis and retroperitoneal fibrosis respectively, and showed severe fibrosis and inflammation with marked lymphoplasmacytic infiltration and IgG4-positive plasma cell infiltration. Hormone replacement therapy with hydrocortisone resulted in marked clinical improvement. Systemic involvement found in this patient possibly corresponded to the new concept of IgG4-associated multifocal systemic fibrosis.