An asymmetric nine-arm star polymer, (polystyrene)3-(poly(4-methoxystyrene))3-(polyisoprene)3 (PS3-PMOS3-PI3) was synthesized, and the details of the structures of its thin films were successfully investigated for the first time by using in situ grazing incidence X-ray scattering (GIXS) with a synchrotron radiation source. Our quantitative GIXS analysis showed that thin films of the star polymer molecules have very complex but highly ordered and preferentially in-plane oriented hexagonal (HEX) structures consisting of truncated PS cylinders and PMOS triangular prisms in a PI matrix. This HEX structure undergoes a partial rotational transformation process at temperatures above 190 °C that produces a 30°-rotated HEX structure; this structural isomer forms with a volume fraction of 23% during heating up to 220 °C and persists during subsequent cooling. These interesting and complex self-assembled nanostructures are discussed in terms of phase separation, arm number, volume ratio, and confinement effects.