• Although radiomics is potentially a promising approach to analyze medical image data, many pitfalls need to be considered to avoid a reproducibility crisis.• There is a translation gap in radiomics research, with many studies being published but so far little to no translation into clinical practice.• Going forward, more studies with higher levels of evidence are needed, ideally also focusing on prospective studies with relevant clinical impact.