In the past decade, anatomical context features have been widely used for cephalometric landmark detection and significant progress is still being made. However, most existing methods rely on handcrafted graphical models rather than incorporating anatomical context during training, leading to suboptimal performance. In this study, we present a novel framework that allows a Convolutional Neural Network (CNN) to learn richer anatomical context features during training. Our key idea consists of the Local Feature Perturbator (LFP) and the Anatomical Context loss (AC loss). When training the CNN, the LFP perturbs a cephalometric image based on prior anatomical distribution, forcing the CNN to gaze relevant features more globally. Then AC loss helps the CNN to learn the anatomical context based on spatial relationships between the landmarks. The experimental results demonstrate that the proposed framework makes the CNN learn richer anatomical representation, leading to increased performance. In the performance comparisons, the proposed scheme outperforms state-of-the-art methods on the ISBI 2015 Cephalometric X-ray Image Analysis Challenge.