Tm,Ho:CaYLuAlO4 (Tm,Ho:CALYLO) crystal has wide emission spectra both for π-polarization and σ-polarization, showing significant potential for the generation of ultrashort pulses. Here, a widely tunable and passively mode-locked laser operation based on Tm,Ho:CALYLO crystal under two polarizations was demonstrated for what we believe to be the first time ever. For π-polarization, a maximum output power of 1.52 W and a tuning range of 255.3 nm were achieved in the continuous wave (CW) regime. In the mode-locked regime, a pulse duration of 68 fs and an average output power of 228 mW were achieved upon GaSb-based semiconductor saturable absorber mirror (SESAM). As for σ-polarization, a broader tuning range of 267.1 nm was realized, leading to the shorter pulse duration of 58 fs at 79.7 MHz repetition rate.