To address the problem of multiple solutions and improve the calculating speed, we construct a tandem architecture consisting of a forward modeling network and an inverse design network. Using this combined network, we inversely design the circular polarization converter and analyze the effect of different design parameters on the prediction accuracy of the polarization conversion rate. The average mean square error of the circular polarization converter is 0.00121 at an average prediction time of 1.56×10-2 s. If only the forward modeling process is considered, it takes 6.15×10-4 s, which is 2.1×105 times faster than that using the traditional numerical full-wave simulation method. By slightly resizing the network input and output layers, the network is adaptable to the design of both the linear cross-polarization and linear-to-circular polarization converters.