Photoluminescent gold nanoclusters are widely seen as a promising candidate for applications in biosensing and bioimaging. Although they have many of the required properties, such as biocompatibility and photostability, the luminescence of near infrared emitting gold nanoclusters is still relatively weak compared to the best available fluorophores. This study contributes to the ongoing debate on the possibilities and limitations of improving the performance of gold nanoclusters by combining them with plasmonic nanostructures. We focus on a detailed description of the emission enhancement and compare it with the excitation enhancement obtained in recent works. We prepared a well-defined series of gold nanoclusters attached to gold nanorods whose plasmonic band is tuned to the emission band of gold nanoclusters. In the resultant single-element hybrid nanostructure, the gold nanorods control the luminescence of gold nanoclusters in terms of its spectral position, polarization and lifetime. We identified a range of parameters which determine the mutual interaction of both particles including the inter-particle distance, plasmon-emission spectral overlap, dimension of gold nanorods and even the specific position of gold nanoclusters attached on their surface. We critically assess the practical and theoretical photoluminescence enhancements achievable using the above strategy. Although the emission enhancement was generally low, the observations and methodology presented in this study can provide a valuable insight into the plasmonic enhancement in general and into the photophysics of gold nanoclusters. We believe that our approach can be largely generalized for other relevant studies on plasmon enhanced luminescence.