Mass is the most fundamental parameter of a star, yet it is also one of the most difficult to measure directly. In general, astronomers estimate stellar masses by determining the luminosity and using the 'mass-luminosity' relationship, but this relationship has never been accurately calibrated for young, low-mass stars and brown dwarfs. Masses for these low-mass objects are therefore constrained only by theoretical models. A new high-contrast adaptive optics camera enabled the discovery of a young (50 million years) companion only 0.156 arcseconds (2.3 au) from the more luminous (> 120 times brighter) star AB Doradus A. Here we report a dynamical determination of the mass of the newly resolved low-mass companion AB Dor C, whose mass is 0.090 +/- 0.005 solar masses. Given its measured 1-2-micrometre luminosity, we have found that the standard mass-luminosity relations overestimate the near-infrared luminosity of such objects by about a factor of approximately 2.5 at young ages. The young, cool objects hitherto thought to be substellar in mass are therefore about twice as massive, which means that the frequency of brown dwarfs and planetary mass objects in young stellar clusters has been overestimated.