We propose the relaxation of geometries throughout chemical compound space using alchemical perturbation density functional theory (APDFT). APDFT refers to perturbation theory involving changes in nuclear charges within approximate solutions to Schrödinger's equation. We give an analytical formula to calculate the mixed second order energy derivatives with respect to both nuclear charges and nuclear positions (named "alchemical force") within the restricted Hartree-Fock case. We have implemented and studied the formula for its use in geometry relaxation of various reference and target molecules. We have also analyzed the convergence of the alchemical force perturbation series as well as basis set effects. Interpolating alchemically predicted energies, forces, and Hessian to a Morse potential yields more accurate geometries and equilibrium energies than when performing a standard Newton-Raphson step. Our numerical predictions for small molecules including BF, CO, N2, CH4, NH3, H2O, and HF yield mean absolute errors of equilibrium energies and bond lengths smaller than 10 mHa and 0.01 bohr for fourth order APDFT predictions, respectively. Our alchemical geometry relaxation still preserves the combinatorial efficiency of APDFT: Based on a single coupled perturbed Hartree-Fock derivative for benzene, we provide numerical predictions of equilibrium energies and relaxed structures of all 17 iso-electronic charge-neutral BN-doped mutants with averaged absolute deviations of ∼27 mHa and ∼0.12 bohr, respectively.