When an unknown example, one that was not seen during training, appears, most recognition systems usually produce overgeneralized results and determine that the example belongs to one of the known classes. To address this problem, teacher-explorer-student (T/E/S) learning, which adopts the concept of open set recognition (OSR) to reject unknown samples while minimizing the loss of classification performance on known samples, is proposed in this study. In this novel learning method, the overgeneralization of deep-learning classifiers is significantly reduced by exploring various possibilities for unknowns. The teacher network extracts hints about unknowns by distilling the pretrained knowledge about knowns and delivers this distilled knowledge to the student network. After learning the distilled knowledge, the student network shares its learned information with the explorer network. Next, the explorer network shares its exploration results by generating unknown-like samples and feeding those samples to the student network. As this alternating learning process is repeated, the student network experiences a variety of synthetic unknowns, reducing overgeneralization. The results of extensive experiments show that each component proposed in this article significantly contributes to improving OSR performance. It is found that the proposed T/E/S learning method outperforms current state-of-the-art methods.