The frequency stability of lasers is limited by thermal noise in state-of-the-art frequency references. Further improvement requires operation at cryogenic temperature. In this context, we investigate a fiber-based ring resonator. Our system exhibits a first-order temperature-insensitive point around 3.55K, much lower than that of crystalline silicon. The observed low sensitivity with respect to vibrations (<5⋅10-11m-1s2), temperature (-22(1)⋅10-9K-2), and pressure changes (4.2(2)⋅10-11mbar-2) makes our approach promising for future precision experiments.