Vehicle re-identification (re-ID) aims to identify the same vehicle across multiple non-overlapping cameras, which is rather a challenging task. On the one hand, subtle changes in viewpoint and illumination condition can make the same vehicle look much different. On the other hand, different vehicles, even different vehicle models, may look quite similar. In this paper, we propose a novel Two-level Attention network supervised by a Multi-grain Ranking loss (TAMR) to learn an efficient feature embedding for the vehicle re-ID task. The two-level attention network consisting of hard part-level attention and soft pixel-level attention can adaptively extract discriminative features from the visual appearance of vehicles. The former one is designed to localize the salient vehicle parts, such as windscreen and car head. The latter one gives an additional attention refinement at pixel level to focus on the distinctive characteristics within each part. In addition, we present a multi-grain ranking loss to further enhance the discriminative ability of learned features. We creatively take the multi-grain relationship between vehicles into consideration. Thus, not only the discrimination between different vehicles but also the distinction between different vehicle models is constrained. Finally, the proposed network can learn a feature space, where both intra-class compactness and inter-class discrimination are well guaranteed. Extensive experiments demonstrate the effectiveness of our approach and we achieve state-of-the-art results on two challenging datasets, including VehicleID and Vehicle-1M.