Quantum dot (QD)-metal oxide composite forms a "heart" of the QD-sensitized solar cells. It maintains light absorption and electron-hole separation in the system and has been therefore extensively studied. The interest is largely driven by a vision of harvesting the hot carrier energy before it is lost via relaxation. Despite of importance of the process, very little is known about the carrier relaxation in the QD-metal oxide composites. In order to fill this gap of knowledge we carry out a systematic study of initial electron dynamics in different CdSe QD systems. Our data reveal that QD attachment to ZnO induces a speeding-up of transient absorption onset. Detailed analysis of the onset proves that the changes are caused by an additional relaxation channel dependent on the identity of the QD-ZnO linker molecule. The faster relaxation represents an important factor for hot carrier energy harvesting, whose efficiency can be influenced by almost 50%.