The concept of retardance is critically analyzed for ray paths through optical systems described by a three-by-three polarization ray-tracing matrix. Algorithms are presented to separate the effects of retardance from geometric transformations. The geometric transformation described by a "parallel transport matrix" characterizes nonpolarizing propagation through an optical system, and also provides a proper relationship between sets of local coordinates along the ray path. The proper retardance is calculated by removing this geometric transformation from the three-by-three polarization ray-tracing matrix. Two rays with different ray paths through an optical system can have the same polarization ray-tracing matrix but different retardances. The retardance and diattenuation of an aluminum-coated three fold-mirror system are analyzed as an example.