We study the nontrivial clustering properties of extreme or recurrent events in the context of deterministic chaotic systems. We find that correlations between return times of such events can depend nonmonotonically on the threshold used to define the events, which leads to counterintuitive behavior. In particular, the distribution of the conditional return intervals can indicate clustering as well as repelling of extreme events for the same condition but different thresholds-in sharp contrast to what has been observed for stochastic processes with long-range correlations as well as for independent and identically distributed random variables. This has important implications for the time-dependent hazard assessment of extreme events, indicating that possible threshold dependencies should always be taken into account.