A measurement is reported of the jet mass distribution in hadronic decays of boosted top quarks produced in pp collisions at sqrt[s]=13 TeV. The data were collected with the CMS detector at the LHC and correspond to an integrated luminosity of 35.9 fb^{-1}. The measurement is performed in the lepton+jets channel of tt[over ¯] events, where the lepton is an electron or muon. The products of the hadronic top quark decay t→bW→bqq[over ¯]^{'} are reconstructed as a single jet with transverse momentum larger than 400 GeV. The tt[over ¯] cross section as a function of the jet mass is unfolded at the particle level and used to extract a value of the top quark mass of 172.6±2.5 GeV. A novel jet reconstruction technique is used for the first time at the LHC, which improves the precision by a factor of 3 relative to an earlier measurement. This highlights the potential of measurements using boosted top quarks, where the new technique will enable future precision measurements.