It is proven that, within the conventional approach using a common origin and gaugeless basis sets for the calculation of atomic magnetizability and Larmor current density induced by an external magnetic field, the natural gauge origin coincides with the nucleus. Recipes for defining an optimal gauge origin for the calculation of magnetizability and magnetic shielding at the nuclei of a molecule are given. Within the common origin approach, the paramagnetic contributions to the components of magnetic tensors of a molecule are represented by a minimum number of non-vanishing parameters if the gauge origin is chosen at a point characterized by the total molecular symmetry, e.g., the center of electronic charge for magnetizabilities. It is shown that total values of diagonal components of the magnetic shielding tensor σ(I) at a nucleus I in a molecule, as well as separate diamagnetic σ(dI) and paramagnetic σ(pI) contributions, calculated via the common origin method, are origin independent for a number of local point group symmetries. The diagonal components (and the average value) of σ(I) depend on the gauge origin only for nuclear site symmetries C(1), C(s), C(n), C(nv), n = 2, 3.... Group-theoretical methods show interesting features, e.g., for S(4) local symmetry, in a coordinate transformation, the paramagnetic contribution to the zz component and to the trace of the shielding tensor is origin independent, whereas the xx and yy components mix into one another, in such a way that their sum remains constant.