This research aimed to prepare 166Dy₂O₃-iPSMA/166Ho₂O₃-iPSMA nanoparticles (166Dy₂O₃/166Ho₂O₃-iPSMA NPs) and assess the radiation absorbed dose produced by the nanosystem to hepatic cancer cells by using experimental in vitro and in vivo biokinetic data. Dy₂O₃NPs were synthesized and functionalized with the prostate-specific membrane antigen inhibitor peptide (iPSMA). Fourier transform infrared (FTIR) spectroscopy, transmission electron microscope (TEM), dynamic light scattering (DSL) and zeta potential analyses indicated the formation of Dy₂O₃-iPSMA NPs (46.11 ± 13.24 nm). After neutron activation, a stable 166Dy₂O₃/166Ho₂O₃- iPSMA nanosystem was obtained, which showed adequate affinity to the PSMA receptor in HepG2 cancer cells (Kd = 9.87 ± 2.27 nM). in vitro studies indicated high 166Dy₂O₃/166Ho₂O₃-iPSMA internalization in cancer cells, with high radiation doses to cell nuclei (107 Gy) and cytotoxic effects, resulting in a significant reduction in HepG2 cell viability (decreasing to 2.12 ± 0.31%). After intratumoral administration in mice, the nanosystem biokinetic profile indicated significant retention into the tumoral mass, producing ablative radiation doses (>70 Gy).