We propose an idea to excite localized modes in a photonic crystal (PC) waveguide without ruining the discrete translational symmetry of the lattice. This can be done by arranging dispersive elements having negative permittivity over a desired frequency range into a periodic structure. We demonstrate numerically the realization of a cavity mode inside the air region of a geometrical defectless two-dimensional square-lattice PC consisting of polaritonic cylinders placed in air matrix. The corresponding waveguide structure in the form of a PC fiber supports the cavity mode as a guided mode to propagate along the guiding direction at very small propagation constant with near zero group velocity. These localized modes can be recognized as localized defectless modes inside the structure with four-fold symmetry.