In sequential competitions, the order in which teams take turns may have an impact on performance and the outcome. Previous studies with penalty shootouts have shown mixed evidence of a possible advantage for the first shooting team. This has led to some debate on whether a change in the rules of the game is needed. This work contributes to the debate by collecting an extensive dataset of shootouts which corroborates an advantage for the first shooter, albeit with a smaller effect than what has been documented in previous research. To evaluate the impact of alternative ordering of shots, we model shootouts as a probability network, calibrate it using the data from the traditional ordering, and use the model to conduct counterfactual analysis. Our results show that alternating the team that shoots first in each round would reduce the impact of ordering. These results were in part developed as supplement to field studies to support the International Football Association Board's (IFAB) consideration of changing the shooting order.