The prediction of hygroscopic swelling of flexible polymer substrates is crucial in various fields from smart structures to flexible electronics. In this study, the prediction method for time-dependent hygroscopic deformation is presented by employing the finite element method (FEM). In order to precisely consider the strain gradients inside the substrate, moisture distribution depending on time is quantitatively investigated by a moisture absorption analysis and sequentially combined with a mechanical deformation analysis. The essential hygroscopic properties including the saturated moisture content, moisture diffusivity, and the coefficient of moisture expansion are precisely measured. Through the application of these hygroscopic properties to a hygro-mechanical analysis model, the moisture distribution and the hygroscopic deformation are quantitatively simulated with time. For the verification of this model, the simulation results of bilayer structures are compared with the experimental results, which are measured using a three-dimensional deformation measurement system. The presented model demonstrates that the global and local hygroscopic deformations are accurately predicted by this approach, showing above 90% averaged accuracy at each time step. These results can be obtained by precisely measured hygroscopic properties and the consideration of the effect of non-uniform distribution on the hygroscopic deformation.