A nearly nonempirical, transferable model potential is developed for the chlorobenzene molecules (C6ClnH6-n, n = 1 to 6) with anisotropy in the atom-atom form of both electrostatic and repulsion interactions. The potential is largely derived from the charge densities of the molecules, using a distributed multipole electrostatic model and a transferable dispersion model derived from the molecular polarizabilities. A nonempirical transferable repulsion model is obtained by analyzing the overlap of the charge densities in dimers as a function of orientation and separation and then calibrating this anisotropic atom-atom model against a limited number of intermolecular perturbation theory calculations of the short-range energies. The resulting model potential is a significant improvement over empirical model potentials in reproducing the twelve chlorobenzene crystal structures. Further validation calculations of the lattice energies and rigid-body k = 0 phonon frequencies provide satisfactory agreement with experiment, with the discrepancies being primarily due to approximations in the theoretical methods rather than the model intermolecular potential. The potential is able to give a good account of the three polymorphs of p-dichlorobenzene in a detailed crystal structure prediction study. Thus, by introducing repulsion anisotropy into a transferable potential scheme, it is possible to produce a set of potentials for the chlorobenzenes that can account for their crystal properties in an unprecedentedly realistic fashion.