Retroviral gag proteins, as well as fragments minimally containing the capsid (CA) and nucleocapsid (NC) subunits of Gag, are able to spontaneously assemble into virus-like particles (VLPs). This occurs in mammalian and bacterial cells as well as in in vitro systems. In every circumstance, nucleic acids are incorporated into the forming particles. Here, we took advantage of an in vitro system for the generation of non-enveloped Mason-Pfizer monkey virus (M-PMV) VLPs derived from a self-assembling CA-NC subunit of Gag. These VLPs were modified through N-terminal extension of CA-NC with short oligopeptides that, after the assembly process, were exposed on the surface of VLPs. The employed N-terminal modifications allowed specific interaction with target cells expressing prostate-specific membrane antigen. Using this system, we were able to incorporate selected siRNA into forming VLPs and deliver it into the cytosol of target cells. In comparison with other viral vectors designed for targeted transgene delivery, this M-PMV VLP system represents the lowest risk of generating virus-associated pathology, as the VLPs do not contain any viral coding sequences and are formed in a cell-free system.