An important result in classical stochastic thermodynamics is the work fluctuation-dissipation relation (FDR), which states that the dissipated work done along a slow process is proportional to the resulting work fluctuations. We show that slowly driven quantum systems violate this FDR whenever quantum coherence is generated along the protocol, and we derive a quantum generalization of the work FDR. The additional quantum terms in the FDR are found to lead to a non-Gaussian work distribution. Fundamentally, our result shows that quantum fluctuations prohibit finding slow protocols that minimize both dissipation and fluctuations simultaneously, in contrast to classical slow processes. Instead, we develop a quantum geometric framework to find processes with an optimal trade-off between the two quantities.