Fluctuations strongly affect the dynamics and functionality of nanoscale thermal machines. Recent developments in stochastic thermodynamics have shown that fluctuations in many far-from-equilibrium systems are constrained by the rate of entropy production via so-called thermodynamic uncertainty relations. These relations imply that increasing the reliability or precision of an engine's power output comes at a greater thermodynamic cost. Here we study the thermodynamics of precision for small thermal machines in the quantum regime. In particular, we derive exact relations between the power, power fluctuations, and entropy production rate for several models of few-qubit engines (both autonomous and cyclic) that perform work on a quantized load. Depending on the context, we find that quantum coherence can either help or hinder where power fluctuations are concerned. We discuss design principles for reducing such fluctuations in quantum nanomachines and propose an autonomous three-qubit engine whose power output for a given entropy production is more reliable than would be allowed by any classical Markovian model.