Fiber Bragg gratings inscribed with a femtosecond laser using the point-by-point (PbP) technique have polarization dependent grating strength (PDGS) and exhibit birefringence. In this paper we quantify the dependence of these two properties on the ellipticity, position in the core and size of the micro-voids at the center of each refractive index modulation. We demonstrate that the effective modal index for type II gratings written with a femtosecond laser using the PbP method must be lower than that of the pristine fiber, and for the first time associate an axis with a polarization such that the long axis of the elliptically-shaped index modulations corresponds to the slow axis of the gratings. We exploit the PDGS of two gratings used as frequency-selective feedback elements as well as appropriate coiling, to realize a linearly-polarized fiber laser with a low birefringence fiber cavity. We show that the polarization-dependent grating strength is a function of the writing pulse energy and that only gratings optimized for this property will linearly polarize the fiber laser. The fiber lasers have high extinction ratios (>30 dB) for fiber lengths of up to 10 m and very stable polarized output powers (<0.5% amplitude fluctuations) in the range of 20-65 mW at 1540 nm. This method of polarization discrimination allows the realization of highly robust and simplified linearly polarized fiber lasers.