Graphene double quantum dots (DQDs) open to use charge or spin degrees of freedom for storing and manipulating quantum information in this new electronic material. However, impurities and edge disorders in etched graphene nano-structures hinder the ability to control the inter-dot tunnel coupling, tC, the most important property of the artificial molecule. Here we report measurements of tC in an all-metal-side-gated graphene DQD. We find that tC can be controlled continuously about a factor of four by employing a single gate. Furthermore, tC, can be changed monotonically about another factor of four as electrons are gate-pumped into the dot one by one. The results suggest that the strength of tunnel coupling in etched graphene DQDs can be varied in a rather broad range and in a controllable manner, which improves the outlook to use graphene as a base material for qubit applications.