Animals are proof that complete cognitive systems can be realized in neural substrates. It is thus natural that engineers from AI and machine learning have tried to design advanced cognitive systems on the basis of artificial neural networks. This has led to illuminating concepts and architectures in fields like computational linguistics, dynamic pattern recognition, autonomous agents, or evolutionary robotics. However, if one takes a close and critical look, one finds that nowhere do artificial systems close to biological levels of performance. One important cause for this gap is a lack of appropriate mathematical concepts. Biological neural systems are high-dimensional, nonlinear, heterogeneous, multiscale, nonstationary, stochastic, and heavily input-driven - a cocktail of properties which overwhelms current dynamical systems theory. Inasmuch as we do not possess mathematical models for such systems, we cannot understand them; and inasmuch as we do not understand, we cannot engineer. ...