Ultrasonic tracking is a promising technique in indoor object localization. However, limited success has been reported in dynamic orientational and positional ultrasonic tracking for ultrasound (US) probes due to its instability and relatively low accuracy. This article aims at developing an inertial measurement unit (IMU)-assisted ultrasonic tracking system that enables a high accuracy positional and orientational localization. The system was designed with the acoustic pressure field simulation of the transmitter, receiver configuration, position-variant error simulation, and sensor fusion. The prototype was tested in a tracking volume required in typical obstetric sonography within the typical operation speed ranges (slow mode and fast mode) of US probe movement. The performance in two different speed ranges was evaluated against a commercial optical tracking device. The results show that the proposed IMU-assisted US tracking system achieved centimeter-level positional tracking accuracy with the mean absolute error (MAE) of 12 mm and the MAE of orientational tracking was less than 1°. The results indicate the possibility of implementing the IMU-assisted ultrasonic tracking system in US probe localization.