Pentagonal palladium diselenide (PdSe2) stands out for its exceptional optoelectronic properties, including high carrier mobility, tunable bandgap, and anisotropic electronic and optical responses. Herein, we systematically investigate photocarrier dynamics in PdSe2 ribbons using polarization-resolved optical pump-probe spectroscopy. In thin PdSe2 ribbons with a semiconductor phase, the photocarrier dynamics are found to be dominated by intraband hot-carrier cooling, interband recombination, and the exciton effect, showing weak crystalline orientation dependences. Conversely, in thick semimetal-phase PdSe2 ribbons, the photocarrier relaxations governed by the electron-optical/acoustic phonon scattering strongly depend on the sample orientation, albeit with a degradation in in-plane anisotropy following hot-carrier cooling. Furthermore, we analyze the correlations between photocarrier dynamics and anisotropic energy dispersions of electronic structures across a wide range in k space, as well as the contributions from the anisotropic electron-phonon couplings. Our study provides crucial insights for developing polarization-sensitive photoelectronic devices based on PdSe2.