Thyroid cancer is the most pervasive disease in the endocrine system and is getting extensive attention. The most prevalent method for an early check is ultrasound examination. Traditional research mainly concentrates on promoting the performance of processing a single ultrasound image using deep learning. However, the complex situation of patients and nodules often makes the model dissatisfactory in terms of accuracy and generalization. Imitating the diagnosis process in reality, a practical diagnosis-oriented computer-aided diagnosis (CAD) framework towards thyroid nodules is proposed, using collaborative deep learning and reinforcement learning. Under the framework, the deep learning model is trained collaboratively with multiparty data; afterward classification results are fused by a reinforcement learning agent to decide the final diagnosis result. Within the architecture, multiparty collaborative learning with privacy-preserving on large-scale medical data brings robustness and generalization, and diagnostic information is modeled as a Markov decision process (MDP) to get final precise diagnosis results. Moreover, the framework is scalable and capable of containing more diagnostic information and multiple sources to pursue a precise diagnosis. A practical dataset of two thousand thyroid ultrasound images is collected and labeled for collaborative training on classification tasks. The simulated experiments have shown the advancement of the framework in promising performance.