Mechanistic dynamic models have become an essential tool for understanding biomolecular networks and other biological systems. Biochemical stochasticity can be extremely important in some situations, e.g., at the single-cell level where there is a low copy number of the species involved. In these scenarios, deterministic models are not suitable to characterize the dynamics, so stochastic dynamic models should be considered. Here, we address the challenging problem of parameter estimation in stochastic dynamic models. Despite recent advances, this area is considerably less mature than its deterministic counterpart. We present a novel strategy based on two components: (i) global optimization via a hybrid stochastic-deterministic approach, and (ii) stochastic simulation techniques tailored to the sparsity of the available experimental data. Regarding the latter, for cases of dense population data we make use of a novel approach using a Partial Integro-Differential Equation (PIDE) model solved using a semilagrangian method. In order to further speed up the simulations, we also present efficient parallel implementations for multi-core CPUs and also for graphical processing units (GPUs). Importantly, whereas SDE and Fokker Planck approximations of the Chemical Master Equation (CME) apply when the reactant populations are sufficiently large, the PIDE approximation to the CME is valid for very low copy numbers, and therefore they enable us to tackle parameter estimation for systems with large intrinsic molecular noise, (highly stochastic regimes far from the thermodynamic limit). We test our strategy with four challenging problems: a Lotka-Volterra system, a polarization system in S. cerevisiae, a genetic toggle switch, and a genetic circadian oscillator. Our method could successfully solve these problems in very reasonable computation times (often a few minutes for the first two problems) using standard low-cost hardware, showing very significant speedups with respect to recent alternative methods. The code used to obtain the results reported here is available at https://doi.org/10.5281/zenodo.5195408.