This paper develops an imaging based three-color pyrometer for the monitoring of temperature distribution in a continuous casting billet. A novel optical device, together with an embedded electronic system, is designed to sequentially collect a dark image and three thermal images with specified wavelengths on a same monochromatic charge-coupled-device (CCD). The three thermal images provide the basis for the determination of target temperature, while the dark image is used to online eliminate the dark noise of CCD with a differential method. This image pyrometer is not only independent of target emissivity but also overcomes the dissimilarity of measuring accuracy between the micro-sensors of CCD resulted from the non-uniformity of pixels' intensity response and the vignetting of optical system. Furthermore, a precise two-color temperature field measuring model on the CCD pyrometer is established, based on which a self-adaptive light-integration mechanism is presented. Compared with the traditional fixed light-integration method, the measuring range of the pyrometer is greatly extended and its sensitivity in low temperature segment is improved. The test results in a steel factory demonstrate that the pyrometer is capable of meeting the requirement of surface temperature measurements about casting billets. Reliability and accuracy of measurement results are also discussed herein.