The olive is an important fruit species cultivated for oil and table olives in Italy and the Mediterranean basin. The conservation of cultivated plants in ex situ collections is essential for the optimal management and use of their genetic resources. The largest ex situ olive germplasm collection consists of approximately 500 Italian olive varieties and corresponding to 85% of the total Italian olive germplasm is maintained at the Consiglio per la Ricerca e sperimentazione per l'Agricoltura, Centro di Ricerca per l'Olivicoltura e l'Industria Olearia (CRA-OLI), in Italy. In this work, eleven preselected nuclear microsatellite markers were used to assess genetic diversity, population structure, and gene flows with the aim of assembling a core collection. The dendrogram obtained utilizing the unweighted pair group method highlights the presence of homonymy and synonymy in olive tree datasets analyzed in this study. 439 different unique genotype profiles were obtained with this combination of 11 loci nSSR, representing 89.8% of the varieties analyzed. The remaining 10.2% comprises different variety pairs in which both accessions are genetically indistinguishable. Clustering analysis performed using BAPS software detected seven groups in Italian olive germplasm and gene flows were determined among identified clusters. We proposed an Italian core collection of 23 olive varieties capturing all detected alleles at microsatellites. The information collected in this study regarding the CRA-OLI ex situ collection can be used for breeding programs, for germplasm conservation, and for optimizing a strategy for the management of olive gene pools.