The purpose of the paper is to evaluate the psychometric properties of the impact factor as an assessment procedure. Detailed criteria regarding theoretical underpinnings, test administration, scoring and interpretation are applied. The impact factor appears to be of limited use for deciding which journals to subscribe. It is not suitable for evaluating achievements of individual scientists and research groups. The impact factor contains serious sources of errors and flaws resulting in strong biases against culture- and language-bound medical subspecialties and non-Anglo-American countries. Possible alternatives are discussed.