Anomaly detection plays a crucial role in various real-world applications, including healthcare and finance systems. Owing to the limited number of anomaly labels in these complex systems, unsupervised anomaly detection methods have attracted great attention in recent years. Two major challenges faced by the existing unsupervised methods are as follows: 1) distinguishing between normal and abnormal data when they are highly mixed together and 2) defining an effective metric to maximize the gap between normal and abnormal data in a hypothesis space, which is built by a representation learner. To that end, this work proposes a novel scoring network with a score-guided regularization to learn and enlarge the anomaly score disparities between normal and abnormal data, enhancing the capability of anomaly detection. With such score-guided strategy, the representation learner can gradually learn more informative representation during the model training stage, especially for the samples in the transition field. Moreover, the scoring network can be incorporated into most of the deep unsupervised representation learning (URL)-based anomaly detection models and enhances them as a plug-in component. We next integrate the scoring network into an autoencoder (AE) and four state-of-the-art models to demonstrate the effectiveness and transferability of the design. These score-guided models are collectively called SG-Models. Extensive experiments on both synthetic and real-world datasets confirm the state-of-the-art performance of SG-Models.