Structural hydrological connectivity has been proposed to describe the geological structure of the landscape as well as to explain hydrological behaviors. Indices based on the topological or soil condition were developed to interpret their relationships. While previous studies mainly focused on well-instrumented catchments which are narrow in humidity or temperate zone, the hydrological responses to structural connectivity at the plot and hill slope scale as well as in arid or semi-arid climate conditions remain unclear. This study was conducted in the semi-arid mountainous region of northern China in Haihe Basin which is the source of water of about 350 million people. Experiments were conducted during the rainy season in 2012 and 2013 using four runoff plots. Two indices, flow path length (FL) based on topography and integral connectivity scale length (ICSL) based on soil moisture conditions, developed to represent hydrological connectivity structure and the runoff response to rainfall were analyzed. The results showed that the surface runoff coefficient was strongly and positively linearly correlated to FL, and the correlation between subsurface flow and ICSLs was quadratic. Plots with shorter FL required more rainfall to generate surface runoff. In the shallow soil layer, when the ICSLs are relatively low, the soil can store more water and less rainfall feeds subsurface runoff. Further analysis indicated that improved shallow soil connectivity conditions might enhance the water-holding capacity and lead to lower water yields for each event. This study demonstrated that hydrological structure connectivity could explain the mechanism of runoff generation in semi-arid areas while further experiments should be undertaken to find the threshold-like relationship between FL and surface runoff as well as the influence of plant cover on hydrological behaviors.