Supramolecular polymer networks are three-dimensional structures of crosslinked macromolecules connected by transient, non-covalent bonds; they are a fascinating class of soft materials, exhibiting properties such as stimuli-responsiveness, self-healing, and shape-memory. This critical review summarizes the current state of the art in the physical-chemical characterization of supramolecular networks and relates this knowledge to that about classical, covalently jointed and crosslinked networks. We present a separate focus on the formation, the structure, the dynamics, and the mechanics of both permanent chemical and transient supramolecular networks. Particular emphasis is placed on features such as the formation and the effect of network inhomogeneities, the manifestation of the crosslink relaxation dynamics in the macroscopic sample behavior, and the applicability of concepts developed for classical polymer melts, solutions, and networks such as the reptation model and the principle of time-temperature superposition (263 references).