Cells can sense and respond to physical properties of their surrounding extracellular matrix. We have demonstrated here that tyrosine phosphatase Shp2 plays an essential role in the response of mouse embryonic fibroblasts to matrix rigidity. On rigid surfaces, large focal adhesions (FAs) and anisotropically oriented stress fibers are formed, whereas cells plated on compliant substrates form numerous small FAs and radially oriented stress fibers. As a result, traction force is increased and organized to promote cell spreading and elongation on rigid substrates. Shp2-deficient cells do not exhibit the stiffness-dependent increase in FA size and polarized stress fibers nor the intracellular tension and cell shape change. These results indicate the involvement of Shp2 in regulating the FAs and the cytoskeleton for force maintenance and organization. The defect of FA maturation in Shp2-deficient cells was rescued by expressing Y722F Rho-associated protein kinase II (ROCKII), suggesting that ROCKII is the molecular target of Shp2 in FAs for the FA maturation. Thus, Shp2 serves as a key mediator in FAs for the regulation of structural organization and force orientation of mouse embyonic fibroblasts in determining their mechanical polarity in response to matrix rigidity.