We study the effects of symmetry-breaking defects at continuous quantum transitions (CQTs) of homogeneous systems, which may arise from localized external fields coupled to the order-parameter operator. The problem is addressed within renormalization-group (RG) and finite-size scaling frameworks. We consider the paradigmatic one-dimensional quantum Ising models at their CQT, in the presence of defects which break the global Z_{2} symmetry. We show that such defects can give rise to notable critical crossover regimes where the ground-state properties experience substantial and rapid changes, from symmetric conditions to characterization of these crossover phenomena driven by defects. In particular, this is demonstrated by analyzing the ground-state fidelity associated with small changes of the defect strength. Within the critical crossover regime, the fidelity susceptibility shows a power-law divergence when increasing the system size, related to the RG dimension of the defect strength; in contrast, outside the critical defect regime, it remains finite. We support the RG scaling arguments with numerical results.