Quantum interference causes a wavefunction to have sensitive spatial dependence, and this has a significant effect on quantum transport. For example, in a quantum-dot system, the conductance can depend on the lead positions. We investigate, for graphene quantum dots, the conductance variations with the lead positions. Since for graphene the types of boundaries, e.g., zigzag and armchair, can fundamentally affect the quantum transport characteristics, we focus on rectangular graphene quantum dots, for which the effects of boundaries can be systematically studied. For both zigzag and armchair horizontal boundaries, we find that changing the positions of the leads can induce significant conductance variations. Depending on the Fermi energy, the variations can be either regular oscillations or random conductance fluctuations. We develop a physical theory to elucidate the origin of the conductance oscillation/fluctuation patterns. In particular, quantum interference leads to standing-wave-like-patterns in the quantum dot which, in the absence of leads, are regulated by the energy-band structure of the corresponding vertical graphene ribbon. The observed 'coexistence' of regular oscillations and random fluctuations in the conductance can be exploited for the development of graphene-based nanodevices.