In the majority of current radiation therapy (RT) applications, image quality is still assessed subjectively or by utilizing physical measures. A novel theory that applies objective task-based image quality assessment in radiation therapy (IQA-in-RT) was recently proposed, in which the area under the therapeutic operating characteristic curve (AUTOC) was employed as the figure-of-merit (FOM) for evaluating RT effectiveness. Although theoretically more appealing than conventional subjective or physical measures, a comprehensive implementation and evaluation of this novel task-based IQA-in-RT theory is required for its further application in improving clinical RT. In this work, a practical and modular IQA-in-RT framework is presented for implementing this theory for the assessment of imaging components on the basis of RT treatment outcomes. Computer-simulation studies are conducted to demonstrate the feasibility and utility of the proposed IQA-in-RT framework in optimizing x-ray computed tomography (CT) pre-treatment imaging, including the optimization of CT imaging dose and image reconstruction parameters. The potential advantages of optimizing imaging components in the RT workflow by use of the AUTOC as the FOM are also compared against those of other physical measures. The results demonstrate that optimization using the AUTOC leads to selecting different parameters from those indicated by physical measures, potentially improving RT performance. The sources of systemic randomness and bias that affect the determination of the AUTOC are also analyzed. The presented work provides a practical solution for the further investigation and analysis of the task-based IQA-in-RT theory and advances its applications in improving RT clinical practice and cancer patient care.