The distance between a straight line and a straight line segment in the image space is proposed in this paper. Based on this distance, the neighborhood of a straight line segment is defined and mapped into the parameter space to obtain the parameter space neighborhood of the straight line segment. The neighborhood mapping between the image space and parameter space is a one to one reversible map. The mapped region in the parameter space is analytically derived and it is proved that it can be efficiently approximated by a quadrangle. The proposed straight line segment neighborhood technique for the HT outperforms conventional straight line neighborhood methods currently used with existing HT variations. In contrast to the straight line neighborhoods used in existing HT variations, the proposed straight line segment neighborhood has several advantages including: 1) the detection error of the proposed neighborhood is not affected by the length of the straight line segments; 2) a precision requirement in the image space described using the proposed distance can be explicitly resolved using the proposed formulation; 3) the proposed neighborhood has the ability to distinguish between segments belonging to the same straight line. A variety of experiments are executed to demonstrate that the proposed neighborhood has a variety of interesting properties of high practical value.