Patients with extra-adrenal malignancies are diagnosed increasingly with benign adrenal tumors, as well as non-oncology subjects. We aimed to demonstrate the natural course of adrenal adenomas in terms of mass size and hormonal status in oncology and non-oncology subjects. We also compared the characteristics and behavior of adrenal adenomas with adrenal malignancies. In our registry of adrenal tumors (n = 335), we prospectively evaluated 29 oncology subjects (EAM+) and age, gender, and follow-up duration matched 110 non-oncology subjects (EAM-) with adrenal adenomas. Median follow-up was 24 months. We also included 16 subjects with adrenal malignancies (primary; 3 and metastasis; 13). Tumor size was followed-up with CT or MRI at 6th and 12th months and annually in subsequent visits. Hormonal assessment was repeated at the 6th month after the initial visit and annually in subsequent visits. Initial tumor size, mean increase in tumor size, and number of subjects who showed mass enlargement or developed subclinical Cushing Syndrome were comparable (P > 0.05) between EAM+ and EAM- groups. Subjects with malignant adrenal tumors were older (P = 0.06), had larger tumors at presentation (P < 0.001), and showed mass enlargement during a shorter follow-up duration (P < 0.001). Oncology subjects with adrenal adenomas featured similar baseline and follow-up parameters in terms of mass enlargement and development of subclinical Cushing Syndrome when compared with non-oncology subjects. Malignant adrenal tumors were characterized with large, rapidly growing tumors of older ages. Conservative approach can be suggested to oncology subjects for adrenal adenomas unless clinical and radiological suspicion of adrenal malignancy is present.