We describe an apparatus for the implementation of hybrid optomechanical systems at 4 K. The platform is based on a high-finesse, micrometer-scale fiber Fabry-Perot cavity, which can be widely tuned using piezoelectric positioners. A mechanical resonator can be positioned within the cavity in the object-in-the-middle configuration by a second set of positioners. A high level of stability is achieved without sacrificing either performance or tunability, through the combination of a stiff mechanical design, passive vibration isolation, and an active Pound-Drever-Hall feedback lock incorporating a reconfigurable digital filter. The stability of the cavity length is demonstrated to be better than a few picometers over many hours both at room temperature and at 4 K.