The numerical prediction of acoustofluidic particle motion is of great help for the design, the analysis, and the physical understanding of acoustofluidic devices as it allows for a simple and direct comparison with experimental observations. However, such a numerical setup requires detailed modeling of the acoustofluidic device with all its components and thorough understanding of the acoustofluidic forces inducing the particle motion. In this work, we present a 3D trajectory simulation setup that covers the full spectrum, comprising a time-harmonic device model, an acoustic streaming model of the fluid cavity, a radiation force simulation, and the calculation of the hydrodynamic drag. In order to make quantitatively accurate predictions of the device vibration and the acoustic field, we include the viscous boundary layer damping. Using a semi-analytical method based on Nyborg's calculations, the boundary-driven acoustic streaming is derived directly from the device simulation and takes into account cavity wall vibrations which have often been neglected in the literature. The acoustic radiation forces and the hydrodynamic drag are calculated numerically to handle particles of arbitrary shape, structure, and size. In this way, complex 3D particle translation and rotation inside experimental microdevices can be predicted. We simulate the rotation of a microfiber in an amplitude-modulated 2D field and analyze the results with respect to experimental observations. For a quantitative verification, the motion of an alumina microdisk is compared to a simple experiment. Demonstrating the potential of the simulation setup, we compute the trajectory of a red blood cell inside a realistic microdevice under the simultaneous effects of acoustic streaming and radiation forces.