Smoke removal is an important and meaningful issue for endoscopic surgery, which can enhance the visual quality of endoscopic images. Because it is practically impossible to construct a large training dataset of pair-matched endoscopic images with/without smoke, the Generative Adversarial Nets (GANs) based models are usually used for endoscopic image desmoke. But they have difficulties in either locating the accurate smoke area, or recovering realistic internal organ or tissue details. In this paper, we propose a new approach, called Desmoke-CycleGAN, which combined detection, estimation, and removal of smoke together, to improve the CycleGAN model for endoscopic image smoke removal. In addition, both pixel-level and perceptual-level consistency loss have been incorporated in the proposed model, which helps the model to be more stable and efficient for recovering realistic details in endoscopic images. The experimental results have demonstrated that this method outperforms other state-of-the-art smoke removal approaches with unpaired real endoscopic images.