We study the clustering properties of inertial particles in a turbulent viscoelastic fluid. The investigation is carried out by means of direct numerical simulations of turbulence in the Oldroyd-B model. The effects of polymers on the small-scale properties of homogeneous turbulence are considered in relation with their consequences on clustering of particles, both lighter and heavier than the carrying fluid. We show that, depending on particle and flow parameters, polymers can either increase or decrease clustering.