Extreme geophysical events are of crucial relevance to our daily life: they threaten human lives and cause property damage. To assess the risk and reduce losses, we need to model and probabilistically predict these events. Parametrizations are computational tools used in the Earth system models, which are aimed at reproducing the impact of unresolved scales on resolved scales. The performance of parametrizations has usually been examined on typical events rather than on extreme events. In this paper, we consider a modified version of the two-level Lorenz'96 model and investigate how two parametrizations of the fast degrees of freedom perform in terms of the representation of extreme events. One parametrization is constructed following Wilks [Q. J. R. Meteorol. Soc. 131, 389-407 (2005)] and is constructed through an empirical fitting procedure; the other parametrization is constructed through the statistical mechanical approach proposed by Wouters and Lucarini [J. Stat. Mech. Theory Exp. 2012, P03003 (2012); J. Stat. Phys. 151, 850-860 (2013)]. The two strategies show different advantages and disadvantages. We discover that the agreement between parametrized models and true model is in general worse when looking at extremes rather than at the bulk of the statistics. The results suggest that stochastic parametrizations should be accurately and specifically tested against their performance on extreme events, as usual optimization procedures might neglect them.