Siberian wild rye (Elymus sibiricus L.), an allotetraploid species, is a potentially high-quality perennial forage crop native to temperate regions. We used fluorescently conjugated oligonucleotides, representing ten repetitive sequences, including 6 microsatellite repeats, two satellite repeats, and two ribosomal DNAs, to characterize E. sibiricus chromosomes, using sequential fluorescence in situ hybridization and genomic in situ hybridization assays. Our results showed that microsatellite repeats (AAG)10 or (AGG)10, satellite repeats pAs1 and pSc119.2, and ribosomal 5S rDNA and 45S rDNA are specific markers for unique chromosomes. A referable karyotype ideogram was suggested, by further polymorphism screening, across different E. sibiricus cultivars with a probe mixture of (AAG)10, Oligo-pAs1, and Oligo-pSc119.2. Chromosomal polymorphisms vary between different genomes and between different individual chromosomes. In particular, two distinct forms of chromosome E in H genome were identified in intra- and inter-populations. Here, the significance of these results, for E. sibiricus genome research and breeding, and novel approaches to improve fluorescence in situ hybridization-based karyotyping are discussed.