The design and performance of a low-noise, modular cryogenic probe, which is applicable to a wide range of measurements over a broad range of working frequencies, temperatures, and magnetic fields, is presented. The design of the probe facilitates the exchange of sample holders and sample-stage amplifiers, which, combined with its characteristic low transmission and reflection loss, make this design suitable for high precision or low sensitivity measurements. The specific example of measuring the shot noise of magnetic tunnel junctions is discussed. We highlight various design characteristics chosen specifically to expand the applicability of the probe to measurement techniques such as nuclear magnetic resonance.