Surface-enhanced Raman scattering (SERS) orientational imaging is a recently developed all-optical technique able to determine SERS-active silver nanoparticle dimer orientations by observing lobe positions in SERS emission patterns formed by the directional polarization of SERS along the longitudinal axis of the dimer. Here we extend this technique to discriminate nanoparticle dimers from higher order aggregates by observing the wavelength dependence of SERS emission patterns, which are unchanged in nanoparticle dimers but show differences in higher order aggregates involving two or more nanoparticle junctions. The ability of SERS orientational imaging to identify stacked nanoparticles in higher order aggregates is also demonstrated. The shape of the SERS emission patterns originating from trimers labeled with low and high concentrations of dye is investigated, showing that the emission pattern lobes become less defined as the dye concentration increases. Dynamic fluctuations in the SERS emission pattern lobes are observed in aggregates labeled with low dye concentrations, as molecules diffuse into regions of higher electromagnetic enhancement in multiple nanoparticle junctions.