Over the last years, inertial sensing has proven to be a suitable ambulatory alternative to traditional human motion tracking based on optical position measurement systems, which are generally restricted to a laboratory environment. Besides many advantages, a major drawback is the inherent drift caused by integration of acceleration and angular velocity to obtain position and orientation. In addition, inertial sensing cannot be used to estimate relative positions and orientations of sensors with respect to each other. In order to overcome these drawbacks, this study presents an Extended Kalman Filter for fusion of inertial and magnetic sensing that is used to estimate relative positions and orientations. In between magnetic updates, change of position and orientation are estimated using inertial sensors. The system decides to perform a magnetic update only if the estimated uncertainty associated with the relative position and orientation exceeds a predefined threshold. The filter is able to provide a stable and accurate estimation of relative position and orientation for several types of movements, as indicated by the average rms error being 0.033 m for the position and 3.6 degrees for the orientation.