Fetal monitoring during pregnancy is important to support medical decision making. The fetal electrocardiogram (fECG) is a valuable signal to diagnose fetal well-being. Non-invasive recording of the fECG is performed by positioning electrodes on the maternal abdomen. The signal-to-noise ratio of these recordings is relatively low and the main undesired signal is the maternal electrocardiogram (mECG). Existing methods to remove the mECG signal are not sufficiently accurate to extract the complete fECG signal. In this paper, a novel method for removal of the mECG signal from abdominal recordings is presented. It is an extension of the linear prediction method. Each mECG complex is segmented and these segments are separately estimated by linear prediction. Both the presented method and the standard linear prediction are applied to simulated abdominal recordings and evaluated by determining the rms errors between the estimated and the actual fECG signals. The ratio between the rms errors of the linear prediction method and the presented method varies between 0.4 dB and 2.3 dB. It can therefore be concluded that the presented method is capable of a more accurate removal of the mECG signal for all simulated abdominal recordings with respect to the linear prediction method.