Nonergodicity observed in single-particle tracking experiments is usually modeled by transient trapping rather than spatial disorder. We introduce models of a particle diffusing in a medium consisting of regions with random sizes and random diffusivities. The particle is never trapped but rather performs continuous Brownian motion with the local diffusion constant. Under simple assumptions on the distribution of the sizes and diffusivities, we find that the mean squared displacement displays subdiffusion due to nonergodicity for both annealed and quenched disorder. The model is formulated as a walk continuous in both time and space, similar to the Lévy walk.