Many systems in various scientific fields like medicine, ecology, economics or climate science exhibit so-called critical transitions, through which a system abruptly changes from one state to a different state. Typical examples are epileptic seizures, changes in the climate system or catastrophic shifts in ecosystems. In order to predict imminent critical transitions, a mathematical apparatus called early warning signals has been developed and this method is used successfully in many scientific areas. However, not all critical transitions can be detected by this approach (false negative) and the appearance of early warning signals does not necessarily proof that a critical transition is imminent (false positive). Furthermore, there are whole classes of systems that always show early warning signals, even though they do not feature critical transitions. In this study we identify such classes in order to provide a safeguard against a misinterpretation of the results of an early warning signal analysis of such systems. Furthermore, we discuss strategies to avoid such systematic false positives and test our theoretical insights by applying them to real world data.