We consider closed quantum systems (into which baths may be integrated) that are driven, i.e., subject to time-dependent Hamiltonians. Our point of departure is the assumption that if systems start in non-Gibbsian states at some initial energies, the resulting probability distributions of work may be largely independent of the specific initial energies. It is demonstrated that this assumption has some far-reaching consequences, e.g., it implies the validity of the Jarzynski relation for a large class of non-Gibbsian initial states. By performing numerical analysis on integrable and nonintegrable spin systems, we find the above assumption fulfilled for all examples considered. Through an analysis based on Fermi's golden rule, we partially relate these findings to the applicability of the eigenstate thermalization ansatz to the respective driving operators.