Several prediction models for fatty liver disease (FLD) are available with limited externally validation and less comprehensive evaluation. The aim was to perform external validation and direct comparison of 4 prediction models (the Fatty Liver Index, the Hepatic Steatosis Index, the ZJU index, and the Framingham Steatosis Index) for FLD both in the overall population and the obese subpopulation.This cross-sectional study included 4247 subjects aged 20 to 65 years recruited from the north of Shanxi Province in China. Anthropometric and biochemical features were collected using standard protocols. FLD was diagnosed by liver ultrasonography. We assessed all models in terms of discrimination, calibration, and decision curve analysis.The original models performed well in terms of discrimination for the overall population, with the area under the receiver operating characteristic curves (AUCs) around 0.85, while AUCs for obese individuals were around 0.68. Nevertheless, the predicted risks did not match well with the observed risks both in the overall population and the obese subpopulation. The FLI 2006 was 1 of the 2 best models in terms of discrimination (AUCs were 0.87 and 0.72 for the overall population and the obese subgroup, respectively) and had the best performance in terms of calibration, and attained the highest net benefit.The FLI 2006 is overall the best tool to identify high risk individuals and has great clinical utility. Nonetheless, it does not perform well enough to quantify the actual risk of FLD, which need to be (re)calibrated for clinical use.