Anomaly detection is widely used in many fields to reveal the abnormal process of a system. Typical model-based anomaly detection methods work well in general anomaly detection problems. However, in some application-specific scenarios, the anomalies of interest are "direction-related," that is, only deviation in certain directions of the data space is abnormal. Most existing anomaly detection methods do not work well in these scenarios, especially when there is no abnormal data information during training. Considering that in many real anomaly detection applications such as medical disease detection and industrial device faults diagnosis, the normal data have several ordinal levels, and the anomalies can be regarded as an unseen level distributed roughly along the ordinal direction outside the normal levels. Notice that the ordinal information is inherently "direction-related," and we can use the ordinal information to assist in finding a "direction-related" boundary for the normal data to detect anomalies of interest. A typical type of methods utilizing the ordinal information is ordinal regression. However, to the best of our knowledge, the existing ordinal regression methods are unable to be directly applied to anomaly detection. In this article, to detect the aforementioned "direction-related" anomalies, we propose an ordinal regression algorithm for direction-related anomaly detection (ORAD). Specifically, we first formulate ORAD as an optimization problem. Then, we apply the difference of convex functions (DC) programming to solve the problem to obtain a "direction-related" boundary. After that, we calculate the outlier scores based on the deviation from the boundary. Theoretically, we analyze the ordinal properties and the convergence of ORAD. We carry out experiments on both synthetic data and real datasets to demonstrate the effectiveness of the proposed ORAD.