Due to their relative ease of handling and low cost, inertial measurement unit (IMU)-based joint angle measurements are used for a widespread range of applications. These include sports performance, gait analysis, and rehabilitation (e.g., Parkinson's disease monitoring or poststroke assessment). However, a major downside of current algorithms, recomposing human kinematics from IMU data, is that they require calibration motions and/or the careful alignment of the IMUs with respect to the body segments. In this article, we propose a new method, which is alignment-free and self-calibrating using arbitrary movements of the user and an initial zero reference arm pose. The proposed method utilizes real-time optimization to identify the two dominant axes of rotation of the elbow joint. The performance of the algorithm was assessed in an optical motion capture laboratory. The estimated IMU-based angles of a human subject were compared to the ones from a marker-based optical tracking system. The self-calibration converged in under 9.5 s on average and the rms errors with respect to the optical reference system were 2.7° for the flexion/extension and 3.8° for the pronation/supination angle. Our method can be particularly useful in the field of rehabilitation, where precise manual sensor-to-segment alignment as well as precise, predefined calibration movements are impractical.