We have investigated passive mode-locking of Tm,Ho:YAG lasers with GaInAs- and GaSb-based semiconductor saturable absorber mirrors (SESAMs). With a GaInAs-based SESAM, stable dual-wavelength mode-locking operation was achieved at 2091 nm and 2097 nm, generating pulses with duration of 56.9 ps and a maximum output power of 285 mW. By using the GaSb-based SESAMs, we could generate mode-locked pulses as short as 21.3 ps at 2091 nm with a maximum output power of 63 mW. We attribute the shorter pulse duration obtained with the GaSb SESAMs to the ultrafast recovery time of the absorption and higher nonlinearity compared to standard GaInAs SESAMs.