The formation of GaSb nanopillars by low energy ion sputtering is studied in real-time by spectroscopic Mueller matrix ellipsometry, from the initial formation in the smooth substrate until nanopillars with a height of 200-300 nm are formed. As the nanopillar height increased above 100 nm, coupling between orthogonal polarization modes was observed. Ex situ angle resolved Mueller polarimetry measurements revealed a 180° azimuth rotation symmetry in the off-diagonal Mueller elements, which can be explained by a biaxial material with different dielectric functions εx and εy in a plane parallel to the substrate. This polarization coupling can be caused by a tendency for local direction dependent alignment of the pillars, and such a tendency is confirmed by scanning electron microscopy. Such observations have not been made for GaSb nanopillars shorter than 100 nm, which have optical properties that can be modeled as a uniaxial effective medium.