Semiconductor spin qubits have recently seen major advances in coherence time and control fidelities, leading to a single-qubit performance that is on par with other leading qubit platforms. Most of this progress is based on microwave control of single spins in devices made of isotopically purified silicon. For controlling spins, the exchange interaction is an additional key ingredient which poses new challenges for high-fidelity control. Here, we demonstrate exchange-based single-qubit gates of two-electron spin qubits in GaAs double quantum dots. Using careful pulse optimization and closed-loop tuning, we achieve a randomized benchmarking fidelity of (99.50±0.04)% and a leakage rate of 0.13% out of the computational subspace. These results open new perspectives for microwave-free control of singlet-triplet qubits in GaAs and other materials.