Assay interference is an acknowledged problem in high-throughput screening, and pan-assay interference compounds (PAINS) filters are one of a number of approaches that have been suggested for identification of potential screening artifacts or frequent hitters. Many studies have highlighted that the unwary usage of these structural alerts should be reconsidered and criticized their extrapolation beyond the applicability domain. A large-scale investigation of the activity profiles and the structural context of PAINS might provide a better assessment of whether this extrapolation is valid. To this end, multiple publicly accessible compound collections were screened, and the PAINS statistics are comprehensively presented and discussed. Next, the promiscuity trends and activity profiles of PAINS were compared with those compounds not matching any PAINS substructures. Overall, PAINS demonstrated higher promiscuity and relatively higher assay hit rates compared with the other compounds. Furthermore, nearly 2000 distinct target-ligand complexes containing PAINS were analyzed, and the interactions were quantified and compared. In more than 50% of the instances, the PAINS atoms participated in interactions more frequently compared with the remaining atoms of the ligand structure. Many PAINS participated in crucial interactions that were often responsible for binding of the ligand, which reaffirms their distinction from those responsible for assay interference. In conclusion, we reinforce that while it is important to employ compound filters to eliminate nonspecific hits, establishing a set of statistically significant and validated PAINS filters is essential to restrain the black-box practice of triaging screening hits matching any of the proposed 480 alerts.