Initial images of Venus's south pole by the Venus Express mission have shown the presence of a bright, highly variable vortex, similar to that at the planet's north pole. Using high-resolution infrared measurements of polar winds from the Venus Express Visible and Infrared Thermal Imaging Spectrometer (VIRTIS) instrument, we show the vortex to have a constantly varying internal structure, with a center of rotation displaced from the geographic south pole by ~3 degrees of latitude and that drifts around the pole with a period of 5 to 10 Earth days. This is indicative of a nonsymmetric and varying precession of the polar atmospheric circulation with respect to the planetary axis.