In general we are interested in dynamical systems coupled to complex hysteresis. Therefore as a first step we investigated recently the dynamics of a periodically driven damped harmonic oscillator coupled to independent Ising spins in a random field. Although such a system does not produce hysteresis, we showed how to characterize the dynamics of such a piecewise-smooth system, especially in the case of a large number of spins [Zech, Otto, and Radons, Phys. Rev. E 101, 042217 (2020)2470-004510.1103/PhysRevE.101.042217]. In this paper we extend our model to spin dimers, thus pairwise interacting spins. We show in which cases two interacting spins can show elementary hysteresis, and we give a connection to the Preisach model, which allows us to consider an infinite number of spin pairs. This thermodynamic limit leads us to a dynamical system with an additional hysteretic force in the form of a generalized play operator. By using methods from general chaos theory, piecewise-smooth system theory, and statistics we investigate the chaotic behavior of the dynamical system for a few spins and also in the case of a larger number of spins by calculating bifurcation diagrams, Lyapunov exponents, fractal dimensions, and self-averaging properties. We find that the fractal dimensions and the magnetization are in general not self-averaging quantities. We show how the dynamical properties of the piecewise-smooth system for a large number of spins differs from the system in its thermodynamic limit.