Retrospective detection of Adverse Drug Events (ADEs) is challenging, notably because ADEs result from complex interactions between many factors. Data mining techniques have recently emerged in the field of automated retrospective ADE detection. The "ADE Scorecards" are a research application based on data-mining that has been built in the framework of the PSIP European Project, and potentially enables automated ADE retrospective detection. The objective of this paper is to evaluate the use of the ADE Scorecards in a real-life healthcare situation. For that purpose, the ADE Scorecards have been implemented in a French general hospital and have been used by the physicians and pharmacists for three years (corresponding to 73,000 inpatient stays). According to the results, 2% of the analyzed inpatient stays have a potential ADE with hyperkalemia, and 1% of them have a potential ADE with vitamin K antagonist overdose. In practice, the application, which was first designed to be a standalone web-based application for the physicians, has been used as a part of a more global quality improvement approach led by the pharmacists.