The mean-square displacement (MSD) is widely utilized to study the dynamical properties of stochastic processes. The time-averaged MSD (TAMSD) provides some information on the dynamics which cannot be extracted from the ensemble-averaged MSD. In particular, the relative standard deviation (RSD) of the TAMSD can be utilized to study the long-time relaxation behavior. In this work, we consider a class of Langevin equations which are multiplicatively coupled to time-dependent and fluctuating diffusivities. Various interesting dynamics models such as entangled polymers and supercooled liquids can be interpreted as the Langevin equations with time-dependent and fluctuating diffusivities. We derive a general formula for the RSD of the TAMSD for the Langevin equation with the time-dependent and fluctuating diffusivity. We show that the RSD can be expressed in terms of the correlation function of the diffusivity. The RSD exhibits the crossover at the long time region. The crossover time is related to a weighted average relaxation time for the diffusivity. Thus the crossover time gives some information on the relaxation time of fluctuating diffusivity which cannot be extracted from the ensemble-averaged MSD. We discuss the universality and possible applications of the formula via some simple examples.