A fully 4D joint-estimation approach to reconstruction of temporal sequences of 3D positron emission tomography (PET) images is proposed. The method estimates both a set of temporal basis functions and the corresponding coefficient for each basis function at each spatial location within the image. The joint estimation is performed through a fully 4D version of the maximum likelihood expectation maximization (ML-EM) algorithm in conjunction with two different models of the mean of the Poisson measured data. The first model regards the coefficients of the temporal basis functions as the unknown parameters to be estimated and the second model regards the temporal basis functions themselves as the unknown parameters. The fully 4D methodology is compared to the conventional frame-by-frame independent reconstruction approach (3D ML-EM) for varying levels of both spatial and temporal post-reconstruction smoothing. It is found that using a set of temporally extensive basis functions (estimated from the data by 4D ML-EM) significantly reduces the spatial noise when compared to the independent method for a given level of image resolution. In addition to spatial image quality advantages, for smaller regions of interest (where statistical quality is often limited) the reconstructed time-activity curves show a lower level of bias and a lower level of noise compared to the independent reconstruction approach. Finally, the method is demonstrated on clinical 4D PET data.