The dissipative quantum dynamics of an anharmonic oscillator coupled to a bath is studied with the purpose of elucidating the differences between the relaxation to a spin bath and to a harmonic bath. Converged results are obtained for the spin bath by the surrogate Hamiltonian approach. This method is based on constructing a system-bath Hamiltonian, with a finite but large number of spin bath modes, that mimics exactly a bath with an infinite number of modes for a finite time interval. Convergence with respect to the number of simultaneous excitations of bath modes can be checked. The results are compared to calculations that include a finite number of harmonic modes carried out by using the multiconfiguration time-dependent Hartree method of Nest and Meyer [J. Chem. Phys. 119, 24 (2003)]. In the weak coupling regime, at zero temperature and for small excitations of the primary system, both methods converge to the Markovian limit. When initially the primary system is significantly excited, the spin bath can saturate restricting the energy acceptance. An interaction term between bath modes that spreads the excitation eliminates the saturation. The loss of phase between two cat states has been analyzed and the results for the spin and harmonic baths are almost identical. For stronger couplings, the dynamics induced by the two types of baths deviate. The accumulation and degree of entanglement between the bath modes have been characterized. Only in the spin bath the dynamics generate entanglement between the bath modes.