Graphene has been considered as a promising material for opto-electronic devices, because of its tunable and wideband optical properties. In this work, we demonstrate electro-refractive phase modulation in graphene at wavelengths from 1530 to 1570 nm. By integrating a gated graphene layer in a silicon-waveguide based Mach-Zehnder interferometer, the key parameters of a phase modulator like change in effective refractive index, insertion loss and absorption change are extracted. These experimentally obtained values are well reproduced by simulations and design guidelines are provided to make graphene devices competitive to contemporary silicon based phase modulators for on-chip applications.