We investigate the Berry phase in the nanocrystal complex made of a metal nanoparticle and a slowly rotating semiconductor quantum dot under the radiation of a circularly polarized light. The Berry phase in the dynamic system is found to be more effective to manifest the interaction between the plasmon in the metal nanoparticle and the exciton in the quantum dot. The dependences of the Berry phase on the interparticle distance and the relative position are studied in the weak field condition. The methods to observe the Berry phase are also given.