The standard definition of the Riemann-Liouville integral is revisited. A new fractional integral is proposed with an exponential kernel. Furthermore, some useful properties such as composition relationship of the new fractional integral and Leibniz integral law are provided. Exact solutions of the fractional homogeneous equation and the non-homogeneous equations are given, respectively. Finally, a finite difference scheme is proposed for solving fractional nonlinear differential equations with exponential memory. The results show the efficiency and convenience of the new fractional derivative.