The main objective of this paper is to indicate how the learning process of a neural control system by backpropagation through time (BTT) is influenced by the structure of the control system, the dynamics of the plant and the parameters of the learning process. Since the analysis of a linear system is far more tractable than the analysis of a nonlinear system, learning control of linear systems is pursued as a step towards the understanding of learning control of nonlinear systems. Very good control performance can be attained for linear control systems by BTT learning. Learning of a feedforward controller is enhanced by inclusion of a feedback controller. The attained control performance is not affected by the sensitivity model of the plant. Application of BTT to the control of a nonlinear musculoskeletal model of the human arm shows that both feedback and feedforward control are necessary for a good performance and that both control modes are effectively adapted in the learning process. A satisfactory control performance can still be achieved with drastically simplified sensitivity models during this learning process.