The dynamics of infectious diseases propagating in populations depends both on human interaction patterns, the contagion process and the pathogenesis within hosts. The immune system follows a circadian rhythm and, consequently, the chance of getting infected varies with the time of day an individual is exposed to the pathogen. The movement and interaction of people also follow 24-hour cycles, which couples these two phenomena. We use a stochastic metapopulation model informed by hourly mobility data for two medium-sized Chinese cities. By this setup, we investigate how the epidemic risk depends on the difference of the clocks governing the population movement and the immune systems. In most of the scenarios we test, we observe circadian rhythms would constrain the pace and extent of disease emergence. The three measures (strength, outward transmission and introduction speeds) are highly correlated with each other. For example of the Yushu City, outward transmission speed and introduction speed are correlated with a Pearson's correlation coefficient of 0.83, and the speeds correlate to strength with coefficients of -0.85 and -0.75, respectively (all have p < 0.05), in simulations with no circadian effect and R0 = 1.5. The relation between the circadian rhythms of the immune system and daily routines in human mobility can affect the pace and extent of infectious disease spreading. Shifting commuting times could mitigate the emergence of outbreaks.