Open-set domain adaptation (OSDA) aims to achieve knowledge transfer in the presence of both domain shift and label shift, which assumes that there exist additional unknown target classes not presented in the source domain. To solve the OSDA problem, most existing methods introduce an additional unknown class to the source classifier and represent the unknown target instances as a whole. However, it is unreasonable to treat all unknown target instances as a group since these unknown instances typically consist of distinct categories and distributions. It is challenging to identify all unknown instances with only one additional class. In addition, most existing methods directly introduce marginal distribution alignment to alleviate distribution shift between the source and target domains, failing to learn discriminative class boundaries in the target domain since they ignore categorical discriminative information in the adaptation. To address these problems, in this article, we propose a novel prototype-based shared-dummy classifier (PSDC) model for the OSDA. Specifically, our PSDC introduces an auxiliary dummy classifier to calibrate the source classifier and simultaneously develops a weighted adaptation procedure to align class-wise prototypes for adaptation. We further design a pseudo-unknown learning algorithm to reduce the open-set risk. Extensive experiments on Office-31, Office-Home, and VisDA datasets show that the proposed PSDC can outperform existing methods and achieve the new state-of-the-art performance. The code will be made public.