The technology of surface-enhanced Raman scattering (SERS) has found many applications and may find more if it can possess both sensitivity and thermal stability. This paper reports a rational design of Ag nanorods to simultaneously achieve two competing goals: the sensitivity and the thermal stability of SERS substrates. The Ag nanorods are designed and synthesized using physical vapor deposition under the condition of glancing angle incidence. The working pressure of the vacuum chamber is controlled so the mean free path of depositing atoms is comparable to the dimension of the chamber, so as to grow Ag nanorods with small diameter, and small but clear separation for optimal SERS sensitivity. Such Ag nanorods are further capped with Al2O3 on their top surfaces to reduce the diffusion-induced coarsening at high temperatures, and thereby to improve the thermal stability for SERS detections. Meanwhile, since the side surfaces of Ag nanorods are not coated with oxides in this approach, the SERS sensitivity is largely preserved while good thermal stability is achieved.