Recently, higher-order topological phases that do not obey the usual bulk-edge correspondence principle have been introduced in electronic insulators and brought into classical systems, featuring in-gap corner or hinge states. In this Letter, using near-field scanning measurements, we show the direct observation of corner states in second-order topological photonic crystal slabs consisting of periodic dielectric rods on a perfect electric conductor. Based on the generalized two-dimensional Su-Schrieffer-Heeger model, we show that the emergence of corner states roots in the nonzero edge dipolar polarization instead of the nonzero bulk quadrupole polarization. We demonstrate the topological transition of two-dimensional Zak phases of photonic crystal slabs by tuning intracell distances between two neighboring rods. We also directly observe in-gap one-dimensional edge states and zero-dimensional corner states in the microwave regime. Our work presents that the photonic crystal slab is a powerful platform to directly observe topological states and paves the way to study higher-order photonic topological insulators.