The purpose of this study was to examine the relationships of physical and perceptual-cognitive factors with agility performance in amateur soccer players. Fifteen male amateur soccer players (age, 24.5 ± 1.9 years) completed a linear-sprint test with splits at 5 m, 10 m, and 30 m, a change-of-direction test of 12 m with 2 pre-planned directional changes of 45° at 2 m and 7 m, and a soccer-specific agility test with same movement pattern as the change-of-direction test but with the inclusion of a human stimulus performing passing movements. Additionally, the perceptual-cognitive deficit (agility performance minus change-of-direction performance) was calculated. In relation to agility performance, linear-sprint performance showed large relationships, which were higher with increasing sprint distance (5 m, r = 0.57; 10 m, r = 0.59; 30 m, r = 0.69), change-of-direction performance a very large relationship (r = 0.77), and the perceptual-cognitive deficit a large relationship (r = 0.55). The findings of this study highlight the relatively high contribution of both physical (i.e., linear-sprint and change-of-direction performance) and perceptual-cognitive factors (i.e., perceptual-cognitive deficit) in relation to soccer-specific agility performance at an amateur level. Consequently, such elements might be recommended to be included in training programs aimed at improving agility performance at this playing level. Moreover, the here introduced perceptual-cognitive deficit allows for a convenient and likewise thorough analysis of agility performance. Future studies should investigate the effects of both physically and perceptual-cognitive oriented training interventions on agility performance, which is considered a key element for success in soccer.