We report on an experimental study of the Lipkin-Meshkov-Glick model of quantum spins interacting at infinite range in a transverse magnetic field, which exhibits a ferromagnetic phase transition in the thermodynamic limit. We use dysprosium atoms of electronic spin J=8, subjected to a quadratic Zeeman light shift, to simulate 2J=16 interacting spins 1/2. We probe the system microscopically using single magnetic sublevel resolution, giving access to the spin projection parity, which is the collective observable characterizing the underlying Z_{2} symmetry. We measure the thermodynamic properties and dynamical response of the system, and we study the quantum critical behavior around the transition point. In the ferromagnetic phase, we achieve coherent tunneling between symmetry-broken states, and we test the link between symmetry breaking and the appearance of a finite order parameter.