Cross-scenario monitoring requires domain generalization (DG) for changed knowledge when auxiliary information is unavailable and only one source scenario is involved. In this article, a latent representation generalizing network (LRGN) is proposed to learn transferable knowledge through generalizing the latent representations for cross-scenario monitoring in perimeter security. LRGN is composed of a sequential-variational generative adversarial network (SVGAN), a coupled SVGAN (Co-SVGAN), and a knowledge-aggregated SVGAN. First, the Co-SVGAN can learn domain-invariant latent representations to model dual-domain joint distribution of background data, which is usually sufficient in the source and target scenarios. Deceptive domain shifts are generated based on the domain-invariant latent representations without auxiliary information. Then, SVGAN models the changing knowledge by estimating the distribution of domain shifts. Furthermore, the knowledge-aggregated SVGAN can transfer the learned domain-invariant knowledge from Co-SVGAN for generalizing the latent representations through approximating the distribution of domain shifts. Accordingly, LRGN is trained by a four-phase optimization strategy for DG through generating target-scenario samples of concerned events based on the generalized latent representations. The feasibility and effectiveness of the proposed method are validated through real-field experiments of perimeter security applications in two scenarios.