We present an experimental approach for cryogenic dielectric measurements on ultrathin insulating films. Based on a coplanar microwave waveguide design, we implement superconducting quarter-wave resonators with inductive coupling, which allows us to determine the real part ε1 of the dielectric function at gigahertz frequencies and sample thicknesses down to a few nanometers. We perform simulations to optimize resonator coupling and sensitivity, and we demonstrate the possibility to quantify ε1 with a conformal mapping technique in a wide sample-thickness and ε1-regime. Experimentally, we determine ε1 for various thin-film samples (photoresist, MgF2, and SiO2) in the thickness regime of nanometer up to micrometer. We find good correspondence with nominative values, and we identify the precision of the film thickness as our predominant error source. Additionally, we present a temperature-dependent measurement for a SrTiO3 bulk sample, using an in situ reference method to compensate for the temperature dependence of the superconducting resonator properties.