The Indian summer monsoon (ISM) shows quasi-rhythmic intraseasonal oscillations (ISO) manifested as alternate 'active' phases of copious rainfall and quiescent phases of 'break'. Within these periodic phases, the daily rainfall shows large variability and exhibits spatiotemporally sporadic extreme rainfall events. The recent decades have witnessed a significant increase in the number of these extreme rainfall events, especially in the quiescent phases. This increase is accompanied by a decreasing trend in the mean monsoon rainfall and a weakening variance of its low-frequency ISO (LF-ISO) cycle. However, any physical link between this apparent paradox of increased extreme rainfall events and weakened slower-time-scale components is not yet reported. Here, using observations and numerical model simulations, we show that the occurrence of extreme rainfall events, primarily in the break phase of an LF-ISO cycle, reduce the intensity of the following active phase by stabilizing the atmosphere. We found that extreme events in a monsoon break leads to a reduction in the vertical shear of zonal winds and an increase in the static stability of the atmosphere in the following break-to-active transition and active phases. These conditions oppose the initiation and development of an active phase and lessen its intensity. This reduces the LF-ISO intensity and mean ISM rainfall.