Our current clinical approach to visual hallucinations is largely derived from work carried out by Georges de Morsier in the 1930s. Now, almost a century after his influential papers, we have the research tools to further explore the ideas he put forward. In this review, we address de Morsier's proposal that visual hallucinations in all clinical conditions have a similar neurological mechanism by comparing structural imaging studies of susceptibility to visual hallucinations in Parkinson's disease, Alzheimer's disease, Dementia with Lewy bodies and schizophrenia. Systematic review of the literature was undertaken using PubMed searches. A total of 18 studies across conditions were identified reporting grey matter differences between patients with and without visual hallucinations. Grey matter changes were categorised into brain regions relevant to current theories of visual hallucinations. The distribution of cortical atrophy supports de Morsier's premise that visual hallucinations are invariably linked to aberrant activity within visual thalamo-cortical networks. Further work is required to determine by what mechanism these networks become predisposed to spontaneous activation, and whether the frontal lobe and hippocampal changes identified are present in all conditions. The findings have implications for the development of effective treatments for visual hallucinations.