Colloidal gold nanoparticles represent technological building blocks which are easy to fabricate while keeping full control of their shape and dimensions. Here, we report on a simple two-step maskless process to assemble gold nanoparticles from a water colloidal solution at specific sites of a silicon surface. First, the silicon substrate covered by native oxide is exposed to a charged particle beam (ions or electrons) and then immersed in a HF-modified solution of colloidal nanoparticles. The irradiation of the native oxide layer by a low-fluence charged particle beam causes changes in the type of surface-terminating groups, while the large fluences induce even more profound modification of surface composition. Hence, by a proper selection of the initial substrate termination, solution pH, and beam fluence, either positive or negative deposition of the colloidal nanoparticles can be achieved.