Organisms need to protect themselves against potential dangers from their surroundings, yet they require constant and intimate interactions with the same environment for their survival. The immune system is instrumental for protection against invading organisms and their toxins. The immune system consists of many cell types and is highly integrated within other tissues. Immune activity is particularly enriched at surfaces that separate the host from its environment, such as the skin and the gastrointestinal tract. This enables protection at sites directly at risk but also enables environmental factors to influence the maturation and function of immune structures and cells. Recent work has indicated that the diet in particular is able to influence the immune system and thus affect the development of inflammatory disease. This review aims to highlight recent work on how external factors, with a focus on those derived from the diet such as vitamin A, can have a direct or indirect deterministic influence on the activity and function of immunity.