Recent advances in Deep Learning have led to the development of supervised models to detect anomalies in medical images such as pneumonia in chest X-rays. Automatic detection of such anomalies can help clinicians with faster decision making and treatment planning for patients. Nonetheless, supervised models require complete labeled training data with all possible labels (i.e., positive and negative), which are cumbersome and expensive to obtain. We propose an adversarial learning-based semi-supervised algorithm for anomaly detection, which requires training data only with a single class (positive or negative). We applied our proposed Generative Adversarial Network architecture to detect anomalies and score pneumonia in chest X-rays and achieved statistically significant improvements compared to previous state-of-the-art generative network and one-class classifiers for anomaly detection.