We analyze thermal convection in a fluid layer confined between isothermal horizontal boundaries at which the tangential component of the fluid stress vanishes. The layer rotates about an oblique, nearly vertical axis. Using a model set of equations for w, the horizontal planform of the vertical velocity component, and psi, a stream function related to a large-scale vertical vorticity field, we describe the instabilities of convection rolls. We show how the usual Küppers-Lortz instability, which leads to a continual precession of the roll pattern, can be suppressed by the oblique rotation vector. Of particular interest is the small-angle instability of rolls, to perturbations in the form of rolls that are almost aligned with the primary rolls; at finite Prandtl number, this instability is not prevented by the horizontal component of the rotation vector, unless this component is sufficiently strong, in which case stability is confined to small-amplitude rolls near the marginal stability boundary. A one-dimensional instability leading to amplitude-modulated rolls is unaffected by the oblique rotation. Numerical simulations of the model equations are presented, which illustrate the instabilities analyzed.