Improving skills for accessing and utilizing diet/nutrition information is important for preventing non-communicable diseases and frailty among elderly people. With the rapidly increasing number of elderly people in the world, promoting nutrition literacy among them is a pivotal health policy for maintaining and promoting health. This article describes the barriers and catalysts of nutrition literacy among elderly Japanese people (aged ≥75 years). A cross-sectional analysis of the responses to a questionnaire administered to 678 study participants (men = 347, women = 331) was conducted. Logistic regression analysis revealed that more men had limited nutrition literacy than did women. After stratification by gender, the limited nutrition literacy group was associated with cognitive difficulty in men and women, visual impairment in men and hearing impairment in women. Lower education level and economic status were associated with limited nutrition literacy among women. Adjusted odds ratios (ORs) controlling for age, education level and economic status, as well as cognitive, visual and hearing function, indicated that informational support [OR = 5.59, 95% confidence interval (95% CI) = 1.28-24.49] and diet/nutrition information obtained from friends of the participants (OR = 2.16, 95% CI = 1.11-4.20) were both associated with adequate nutrition literacy among men, whereas diet/nutrition information from health professionals (OR = 3.96, 95% CI = 1.97-7.95) had a significant relation with adequate nutrition literacy among women. Moreover, in the adequate nutrition literacy group, men were more likely to be overweight (OR = 2.17, 95% CI = 1.20-3.91). These findings suggested that cognitive and sensory functions should be taken into account when diet information is provided to older adults. Enhancement of social networks is also a key point in improving nutrition literacy. An effective intervention to improve nutrition literacy in elderly people, particular old men, warrants further investigation.