Fiber Bragg gratings (FBGs) have been advantageously used to improve the chaotic properties of semiconductor lasers. Though these components are known to be highly sensitive to environmental conditions, feedback phase fluctuations are often neglected. In this work, we experimentally demonstrate that the small variations of the propagation time induced by a simple thermal tuning of the FBG are sufficient to induce significant changes of the laser behavior. We report periodic stability enhancements linked with phase variations and highlight that both phase variation and phase offsets play an important role. Last, we show a good qualitative agreement with simulations based on an expanded version of the Lang-Kobayashi model.