Holographic or diffractive optical components are widely implemented using spatial light modulators within optical tweezers to form multiple, and/or modified traps. We show that by further modifying the hologram design to account for residual aberrations, the fidelity of the focused beams can be significantly improved, quantified by a spot sharpness metric. However, the impact this improvement has on the quality of the optical trap depends upon the particle size. For particle diameters on the order of 1 microm, aberration correction can improve the trap performance metric, which is the ratio of the mean square displacement of a corrected trap to an uncorrected trap, in excess of 25%, but for larger particles the trap performance is not unduly affected by the aberrations typically encountered in commercial spatial light modulators.