Systematic mistakes can be distinguished from other types of mistakes in that they are repeatable and predictable within a given organism, and are not due to uncertainty or lack of information. Here we provide a mathematical definition for the concept of systematic mistakes, which captures the way this concept has been used in the behavioral sciences. We also provide an analytical model of information processing networks that are made of large numbers of components, in analogy to the brain being made of a large number of neurons. We show that, for almost all behavioral tasks, and for a wide range of limitations on the computational complexity of the decision-making network, the best possible decision-makers will make systematic mistakes. This result, together with available empirical evidence, suggests that violations of rationality in humans and animals are consistent with natural selection, as the latter operates under constraints.