Hybrid nano-objects formed by two or more disparate materials are among the most promising and versatile nanosystems. A key parameter in their properties is interaction between their components. In this context we have investigated ultrafast charge separation in semiconductor-metal nanohybrids using a model system of gold-tipped CdS nanorods in a matchstick architecture. Experiments are performed using an optical time-resolved pump-probe technique, exciting either the semiconductor or the metal component of the particles, and probing the light-induced change of their optical response. Electron-hole pairs photoexcited in the semiconductor part of the nanohybrids are shown to undergo rapid charge separation with the electron transferred to the metal part on a sub-20 fs time scale. This ultrafast gold charging leads to a transient red-shift and broadening of the metal surface plasmon resonance, in agreement with results for free clusters but in contrast to observation for static charging of gold nanoparticles in liquid environments. Quantitative comparison with a theoretical model is in excellent agreement with the experimental results, confirming photoexcitation of one electron-hole pair per nanohybrid followed by ultrafast charge separation. The results also point to the utilization of such metal-semiconductor nanohybrids in light-harvesting applications and in photocatalysis.