Membrane permeability of drug molecules plays a significant role in the development of new therapeutic agents. Accordingly, methods to predict the passive permeability of drug candidates during a medicinal chemistry campaign offer the potential to accelerate the drug design process. In this work, we combine the physics-based site identification by ligand competitive saturation (SILCS) method and data-driven artificial intelligence (AI) to create a high-throughput predictive model for the passive permeability of druglike molecules. In this study, we present a comparative analysis of four regression models to predict membrane permeabilities of small druglike molecules; of the tested models, Random Forest was the most predictive yielding an R2 of 0.81 for the independent data set. The input feature vector used to train the developed prediction model includes absolute free energy profiles of ligands through a POPC-cholesterol bilayer based on ligand grid free energy (LGFE) profiles obtained from the SILCS approach. The use of the membrane free energy profiles from SILCS offers information on the physical forces contributing to ligand permeability, while the use of AI yields a more predictive model trained on experimental PAMPA permeability data for a collection of 229 molecules. This combination allows for rapid estimations of ligand permeability at a level of accuracy beyond currently available predictive models while offering insights into the contributions of the functional groups in the ligands to the permeability barrier, thereby offering quantitative information to facilitate rational ligand design.