Medication errors and resulting Adverse Drug Events (ADEs) are an important issue of global healthcare. Within the European PSIP project that aims at developing solutions to improve medication safety, contextualized decision support modules aiming to prevent ADEs are being developed. The objective of this work was to thoroughly validate part of the CDSS (Clinical Decision Support System) and the underlying Knowledge Base, in order to detect incorrect or unclear alerts. We systematically developed a repository of test cases and used them for validation. The development of the test cases showed that there are differences among experts in interpreting the correctness of an alert, and that the clinical context is important when judging whether it is adequate. Overall, validation did not find major errors in the Knoweldge Base, but developed several recommendations for further improvement.