Electron demagnetization in a magnetically expanding plasma, a fundamental process for plasma flow and detachment in magnetic nozzles, is experimentally investigated using a rf plasma source and magnetic nozzle (MN). Measurements of the plasma potential spatial profile reveal an ion-confining potential surface, indicative of the edge of a magnetized plasma, that extends along the outermost magnetic flux surface. The downstream extent of the potential surface scales inversely with a characteristic electron Larmor radius, which agrees with an existing theory [E. Ahedo and M. Merino, Phys. Plasmas 19, 083501 (2012)PHPAEN1070-664X10.1063/1.4739791] for electron demagnetization via finite electron Larmor radius (FELR) effects. These results represent the first experimental evidence of FELR demagnetization, and provide an empirical metric for the significance of FELR effects based on the degree of separation between electron and magnetic flux surfaces. With this metric, a critical magnetic field strength is found that ensures electrons remain magnetized through the MN turning point, thus avoiding the rapid plume divergence associated with premature demagnetization.