Olive (Olea europaea L.) is a major fruit crop in the Mediterranean Basin. Ex-situ olive management is essential to ensure optimal use of genetic resources in breeding programs. The Worldwide Olive Germplasm Bank of Córdoba (WOGBC), Spain, and Marrakech (WOGBM), Morocco, are currently the largest existing olive germplasm collections. Characterization, identification, comparison and authentication of all accessions in both collections could thus provide useful information for managing olive germplasm for its preservation, exchange within the scientific community and use in breeding programs. Here we applied 20 microsatellite markers (SSR) and 11 endocarp morphological traits to discriminate and authenticate 1091 olive accessions belonging to WOGBM and WOGBC (554 and 537, respectively). Of all the analyzed accessions, 672 distinct SSR profiles considered as unique genotypes were identified, but only 130 were present in both collections. Combining SSR markers and endocarp traits led to the identification of 535 cultivars (126 in common) and 120 authenticated cultivars. No significant differences were observed between collections regarding the allelic richness and diversity index. We concluded that the genetic diversity level was stable despite marked contrasts in varietal composition between collections, which could be explained by their different collection establishment conditions. This highlights the extent of cultivar variability within WOGBs. Moreover, we detected 192 mislabeling errors, 72 of which were found in WOGBM. A total of 228 genotypes as molecular variants of 74 cultivars, 79 synonyms and 39 homonyms as new cases were identified. Both collections were combined to define the nested core collections of 55, 121 and 150 sample sizes proposed for further studies. This study was a preliminary step towards managing and mining the genetic diversity in both collections while developing collaborations between olive research teams to conduct association mapping studies by exchanging and phenotyping accessions in contrasted environmental sites.