In this article, several optimization methods of two-electron repulsion integral calculations on a graphic processing unit (GPU) are presented. These methods are based on the investigations of the method presented by McMurchie and Davidson (MD). A new Boys function evaluation method for the GPU calculation is introduced. The series summation, the error function, and the finite sum formula method are combined; thus, good performance on the GPU can be achieved. By taking some theoretical study of the McMurchie-Davidson recurrence relations, three major optimization approaches are derived from the deduction of the general term formula for the Hermite expansion coefficient. The three approaches contain a new form of the Hermite expansion coefficients with corresponding recurrence relations, which is more efficient for one-electron integrals and [ss|∗∗] or [∗∗|ss] type two-electron integrals. In addition, a simple yet efficient new recurrence formula for the coefficient evaluation is derived, which is more efficient both in float operations and memory operations than its original one. In average, the new recurrence relation can save 26% float operations and 37% memory operations. Finally, a common sub-expression elimination (CSE) method is implemented. This CSE method is directly generated from some equalities we discovered from the general term formula other than by computer algebra system software. This optimized method achieved up to 3.09 speedups compared to the original MD method on the GPU and up to 92.75 speedups compared to the GAMESS calculation on the central processing unit.