A physical layer key distribution scheme based on chaotic encryption and signal synchronization is proposed in this paper, which can achieve secure key distribution and enhance the security of an orthogonal frequency division multiplexing based passive optical network (OFDM-PON). The key is embedded into the synchronization header and then encrypted by using chaos. The receiver needs to utilize the correct chaotic parameters to successfully decrypt the synchronization information and extract the key. An experiment is conducted to verify the availability of this method by setting key sequences of various length over different transmission distances. The signals of 35.29 Gb/s are successfully transmitted over 5 km, 15 km and 25 km standard single-mode fiber (SSMF), respectively. It is proved that the proposed scheme is feasible and compatible with the traditional encryption algorithms, and it has almost no effect on the synchronization performance, which can then distribute keys with the sending signals without occupying additional channel resources and enhance the security performance of OFDM-PON simultaneously.