Contemporary work in medical decision support is characterized by a multitude of methods. To investigate their relative strengths and weaknesses, we built four diagnostic expert systems based on different methods (Bayes, case-based classification, heuristic classification) for analysis of the same set of 1254 cases of acute abdominal pain previously documented in a prospective multicenter study. The results of the comparative evaluation indicate that differences in overall performance are relatively small (statistically not significant). The performance depends more on the quality of the knowledge base and the case data than on the inference methods of the expert systems. Methods relying exclusively on empirical knowledge (Bayes, case-based classification) tend to have slightly higher overall performance scores due to a diagnostic bias toward ordinary and common diseases. By contrast, methods operating with expert knowledge (e.g., heuristic classification) perform slightly worse overall, but are more sensitive toward uncommon (serious) diseases.