The Internet of Things (IoT) has propelled the evolution of medical sensing technologies to greater heights. Thus, traditional health systems have been transformed into new data-rich environments. This provides an unprecedented opportunity to develop new analytical methods and tools towards a new paradigm of smart and interconnected health systems. Nevertheless, there are risks pertinent to increasing levels of system connectivity and data accessibility. Cyber-attacks become more prevalent and complex, leading to greater likelihood of data breaches. These events bring sudden disruptions to routine operations and cause the loss of billions of dollars. Adversaries often attempt to leverage models to learn a target's sensitive attributes or extrapolate its inclusion within a database. As healthcare systems are critical to improving the wellbeing of our society, there is an urgent need to protect the privacy of patients and minimize the risk of model inversion attacks. This paper presents a new approach, named Mosaic Gradient Perturbation (MGP), to preserve privacy in the framework of predictive modeling, which meets the requirement of differential privacy while mitigating the risk of model inversion. MGP is flexible in fine-tuning the trade-offs between model performance and attack accuracy while being highly scalable for large-scale computing. Experimental results show that the proposed MGP method improves upon traditional gradient perturbation to mitigate the risk of model inversion while offering greater preservation of model accuracy. The MGP technique shows strong potential to circumvent paramount costs due to privacy breaches while maintaining the quality of existing decision-support systems, thereby ushering in a privacy-preserving smart health system.