To minimize the lattice thermal conductivity in thermoelectrics, strategies typically focus on the scattering of low-frequency phonons by interfaces and high-frequency phonons by point defects. In addition, scattering of mid-frequency phonons by dense dislocations, localized at the grain boundaries, has been shown to reduce the lattice thermal conductivity and improve the thermoelectric performance. Here we propose a vacancy engineering strategy to create dense dislocations in the grains. In Pb1-xSb2x/3Se solid solutions, cation vacancies are intentionally introduced, where after thermal annealing the vacancies can annihilate through a number of mechanisms creating the desired dislocations homogeneously distributed within the grains. This leads to a lattice thermal conductivity as low as 0.4 Wm-1 K-1 and a high thermoelectric figure of merit, which can be explained by a dislocation scattering model. The vacancy engineering strategy used here should be equally applicable for solid solution thermoelectrics and provides a strategy for improving zT.