We introduce concepts of external and internal complexity to analyze the relation between an adaptive system and its environment. We apply this theoretical framework to the construction of models in a cognitive system and the selection between hypotheses through selective observations performed on a data set in a recurrent process and propose a corresponding neural network architecture.