Clinical guidelines translate complex research findings and expert opinion into actionable recommendations. However, the effectiveness of even evidence-based guidelines is rarely tested as a whole in a real clinical environment. We have developed a decision support system for implementing clinical guidelines in a busy pediatric practice. We have added to this system the ability to randomize patients to receive care with or without system support of the guideline or guideline components. The randomization is part of the Arden Syntax that implements the system logic. The result is a relatively effortless process for testing guidelines, as they are implemented, to assure that they are effective. We describe the system and the process by which this guideline evaluation functionality was built in, using two guidelines (asthma management and maternal depression screening) both of which have been applied to thousands of patients to date.