It has been widely accepted that there is a close relationship between the land use type and water quality. There have been some researches on this relationship from the perspective of the spatial configuration of land use in recent years. This study aims to analyze the influence of various land use types on the water quality within the Chaohu Lake Basin based on the water quality monitoring data and RS data from 2000 to 2008, with the small watershed as the basic unit of analysis. The results indicated that there was significant negative correlation between forest land and grassland and the water pollution, and the built-up area had negative impacts on the water quality, while the influence of the cultivated land on the water quality was very complex. Besides, the impacts of the landscape diversity on the indicators of water quality within the watershed were also analyzed, the result of which indicated there was a significant negative relationship between them. The results can provide important scientific reference for the local land use optimization and water pollution control and guidance for the formulation of policies to coordinate the exploitation and protection of the water resource.