This study presents the influence of polymorphism on the magnetic properties of Co5TeO8. This compound with a spinel-like structure [Co2]A[Co3Te]BO8 was synthesized into two polymorphs: one disordered within a cubic Fd3̅m structure, where Co2+ and Te6+ ions are randomly distributed on the octahedral B sites [the disordered polymorph can also be presented as an inverse spinel of the formula Co(Co1.5Te0.5)O4] and the other ordered with a cubic P4332 structure where Co2+ and Te6+ ions are ordered on the B sites. The macroscopic magnetic measurements showed that both polymorphs present a ferrimagnetic ordering, below ∼40 K, and a second transition is also observed at 27 K for the ordered polymorph. Neutron powder diffraction data between room temperature and 1.7 K showed as well the presence of short-range magnetic ordered clusters, which appears for both polymorphs below 200 K. At lower temperature, these short-range orders are transformed into long-range ferrimagnetic orders. Below TC = 40 K, the colinear ferrimagnetic structure of the disordered polymorph is described with the I41/am'd' space group. The ordered polymorph undergoes an incommensurate ferrimagnetic spiral spin ordering below TC1 = 45 K, followed by a second magnetic phase transition at TC2 = 27 K. This last transition is associated with the emergence of an additional ferrimagnetic component and an abrupt change in the magnitude of the magnetic propagation vector k = [0, 0, γ] from γ = 0.086 at T = 30 K to γ ≈ 0.14 in the range between 27 and 1.7 K. The magnetic symmetry of the ordered polymorph is described with the P43(00γ)0 magnetic superspace group. We evidenced that the ordering of Co2+/Te6+ on the B sites changes all of the Co-Co and Co-O distances and thus all JAB, JAA, and JBB exchange interactions, between the A and B sites, which are able to stabilize the incommensurate spin modulation in the ordered polymorph.