Gold nanorods were decorated with thermoresponsive copolymers of tailored architecture and constructed from N-isopropyl acrylamide and acrylamide. The copolymers were prepared via reversible addition-fragmentation chain transfer polymerization (RAFT) and immobilized on the gold nanorod surface taking advantage of the aurophilicity of its inherently formed trithiocarbonate groups. The topology as well as the average molecular weight of the copolymers was altered using either a monofunctional or 3-arm star RAFT agent. Two-dimensional arrays of the self-assembled core-shell nanostructures were fabricated by drop-casting showing tunable interparticle spacings. In a simulated blood fluid, the lower critical solution temperature of the nanohybrids could be modified over a significant temperature range around body temperature by adjusting the copolymer composition, the architecture, and/or the size of the polymer. The intrinsic photothermal properties of the gold nanorods were utilized to trigger particle aggregation by irradiation at 808 nm in the optical window of human tissues. In effect, a new nanohybrid system with remotely controllable aggregation via an external NIR-light stimulus for nanomedical applications was developed.