Holonomic quantum computation uses non-Abelian geometric phases to realize error resilient quantum gates. Nonadiabatic holonomic gates are particularly suitable to avoid unwanted decoherence effects, as they can be performed at high speed. By letting the computational system interact with a structured environment, we show that the scope of error resilience of nonadiabatic holonomic gates can be widened to include systematic parameter errors. Our scheme maintains the geometric properties of the evolution and results in an environment-assisted holonomic quantum map that can mimic the effect of a holonomic gate. We demonstrate that the sensitivity to systematic errors can be reduced in a proof-of-concept spin-bath model.