Establishing the structure-property relationship is an important goal of glassy materials, but it is usually impeded by their disordered structure and non-equilibrium nature. Recent studies have illustrated that secondary (β) relaxation is closely correlated with several properties in a range of glassy materials. However, it has been challenging to identify the pertinent structural features that govern it. In this work, we show that the so-called polyamorphous transition in metallic glasses offers an opportunity to distinguish the structural length scale of β relaxation. We find that, while the glass transition temperature and medium-range orders (MROs) change rapidly across the polyamorphous transition, the intensity of β relaxation and the short-range orders (SROs) evolve in a way similar to those in an ordinary reference glass without polyamorphous transition. Our findings suggest that the MRO accounts mainly for the global stiffening of the materials and the glass transition, while the SRO contributes more to β relaxation per se.