Melatonin is widely abundant in many eukaryotic taxa, including various animal phyla, angiosperms, and unicells. In the bioluminescent dinoflagellate Gonyaulax polyedra, melatonin is produced in concentrations sometimes exceeding those found in the pineal gland, exhibits a circadian rhythm with a pronounced nocturnal maximum, and mimics the short-day response of asexual encystment. Even more efficient as a cyst inducer is 5-methoxyptryptamine (5MT), which is also periodically formed in Gonyaulax. In this unicell, the photoperiodic signal-transduction pathway presumably involves melatonin formation, its deacetylation to 5MT, 5MT-dependent transfer of protons from an acidic vacuole, and cytoplasmic acidification. According to this concept, we observe that cyst formation can be induced by various monoamine oxidase inhibitors and protonophores, that 5MT dramatically stimulates H(+)-dependent bioluminescence and leads to a decrease of cytoplasmic pH, as shown by measurements of dicyanohydroquinone fluorescence. Cellular components from Gonyaulax catalyze the photooxidation of melatonin. Its property of being easily destroyed by light in the presence of cellular catalysts may have been the reason that many organisms have developed mechanisms utilizing this indoleamine as a mediator of darkness. Photooxidative reactions of melatonin, as studied with crude Gonyaulax extracts and, more in detail, with protoporphyrin IX as a catalyst, lead to the formation of N1-acetyl-N2-formyl-5-methoxykynuramine (AFMK) as one of the main products. Photochemical mechanisms involve interactions with a photooxidant cation radical leading to the formation of a melatonyl cation radical, which subsequently combines with a superoxide anion.(ABSTRACT TRUNCATED AT 250 WORDS)