The dependence of Brillouin linewidth and peak frequency on lightwave state of polarization (SOP) due to fiber inhomogeneity in single mode fiber (SMF) is investigated by using Brillouin optical time domain analysis (BOTDA) system. Theoretical analysis shows fiber inhomogeneity leads to fiber birefringence and sound velocity variation, both of which can cause the broadening and asymmetry of the Brillouin gain spectrum (BGS) and thus contribute to the variation of Brillouin linewidth and peak frequency with lightwave SOP. Due to fiber inhomogeneity both in lateral profile and longitudinal direction, the measured BGS is the superposition of several spectrum components with different peak frequencies within the interaction length. When pump or probe SOP changes, both the peak Brillouin gain and the overlapping area of the optical and acoustic mode profile that determine the peak efficiency of each spectrum component vary within the interaction length, which further changes the linewidth and peak frequency of the superimposed BGS. The SOP dependence of Brillouin linewidth and peak frequency was experimentally demonstrated and quantified by measuring the spectrum asymmetric factor and fitting obtained effective peak frequency respectively via BOTDA system on standard step-index SMF-28 fiber. Experimental results show that on this fiber the Brillouin spectrum asymmetric factor and effective peak frequency vary in the range of 2% and 0.06MHz respectively over distance with orthogonal probe input SOPs. Experimental results also show that in distributed fiber Brillouin sensing, polarization scrambler (PS) can be used to reduce the SOP dependence of Brillouin linewidth and peak frequency caused by fiber inhomogeneity in lateral profile, however it maintains the effects caused by fiber inhomogeneity in longitudinal direction. In the case of non-ideal polarization scrambling using practical PS, the fluctuation of effective Brillouin peak frequency caused by fiber inhomogeneity provides another limit of sensing frequency resolution of distributed fiber Brillouin sensor.