A channeled spectropolarimeter can simultaneously obtain intensity, spectral, and polarization information. In the traditional model, the retarders must be oriented at specific angles. However, misalignments of the retarders are inevitable during assembly, and the status of the retarders is sensitive to environmental perturbations, which affects the performance of the channeled spectropolarimeter. In this study, a general channeled spectropolarimeter model was derived, in which the retarder orientations can be arbitrary and unknown. Meanwhile, the system is unaffected by environmental perturbation because it can self-calibrate to avoid fluctuations in the retarder orientations and phase retardations. The effectiveness and robustness of the model were verified through simulations and experiments.