Continuous-time interaction data is usually generated under time-evolving environment. Hawkes processes (HP) are commonly used mechanisms for the analysis of such data. However, typical model implementations (such as e.g., stochastic block models) assume that the exogenous (background) interaction rate is constant, and so they are limited in their ability to adequately describe any complex time-evolution in the background rate of a process. In this paper, we introduce a stochastic exogenous rate Hawkes process (SE-HP) which is able to learn time variations in the exogenous rate. The model affiliates each node with a piecewise-constant membership distribution with an unknown number of changepoint locations, and allows these distributions to be related to the membership distributions of interacting nodes. The time-varying background rate function is derived through combinations of these membership functions. We introduce a stochastic gradient MCMC algorithm for efficient, scalable inference. The performance of the SE-HP is explored on real world, continuous-time interaction datasets, where we demonstrate that the SE-HP strongly outperforms comparable state-of-the-art methods.