An all-fiber integrated photodetector is proposed and demonstrated by assembling a graphene/palladium diselenide (PdSe2) Van der Waals heterostructure onto the endface of a standard optical fiber. A gold film is covered on the heterostructure working as an electrode and a mirror, which reflects back the unabsorbed residual light for further reusage. Owing to the low bandgap of PdSe2, the all-fiber photodetector shows a broadband photoresponse from 650 to 1550 nm with a high photoresponsivity of 6.68×104 AW-1, enabling a low light detection of 42.5 pW. And the fastest temporal response is about 660 µs. Taking advantage of heterostructures, the photodetector can work in self-powered mode with the on/off ratio about 82. These findings provide new strategies for integrating two-dimensional materials into optical fibers to realize integrated all-fiber devices with multi-function applications.