The Imperial College COVID-19 Response Team (ICCRT) concluded in a series of high-profile reports that lockdown had been the most effective non-pharmaceutical intervention in 11 European countries during the initial phase of the corona pandemic. As the ICCRT used a transparent modeling framework, we were able to examine assumptions made in the model. We found that the ICCRT modified the assumptions made in their model as more data became available in a way that maintained the conclusion that lockdown was most effective. These observations suggest that modeling of non-pharmaceutical interventions during an ongoing pandemic must be interpreted with caution as sources of error can be found both in the technical execution of the modeling and the assumptions made. The secondary analysis was made possible only because the ICCRT published their methodology in detail, which is a prerequisite for scientific progress in the pandemic modeling area.