Identification of compounds that can bind to a target protein with high affinity is a nontrivial task in structure-based drug design. Several approaches ranging from simple scoring methods to more computationally demanding methods are usually applied for this purpose. In the current work, we used ligand docking in combination with QM/MM-GBSA, MM-GBSA, and MM-PBSA rescoring to discriminate between active and inactive Myt1 kinase inhibitors. Results show that QM/MM-GBSA rescoring performs better than normal docking scores or MM-GBSA rescoring in classifying active and inactive inhibitors. We also applied QM/MM-GBSA rescoring to estimate the binding affinities of compounds from different virtual screening runs. To prove our approach and to confirm its predictive power, a few compounds which were predicted to be active were purchased and experimentally tested. Among the five selected compounds, three showed significant inhibition of recombinant Myt1. PD-173952, which yielded a favorable QM/MM-GBSA binding free energy, showed a K(i) value of 8.1 nM. In addition, two compounds, PD-180970 and saracatinib, showed inhibition at the low micromolar level. Thus, the developed protocol might be useful for further virtual screening experiments to better discriminate between active and inactive compounds and to further optimize the identified hits.