This study aimed to explore the clinical significance of breast tumor tissue stiffness based on ultrasound elastographic evaluation in clinical breast cancer. Tumor tissue stiffness is mainly regulated by interactions among tumor cells, stromal cells, and extracellular matrix and was recently regarded as a representative feature of tumor microenvironment. Basic research has already revealed that the tumor stiffness can lead to tumor progression; however, little is known about its clinical significance because thus far, no useful modality is available in the clinical setting. We investigated the tumor stiffness by strain elastography in 503 consecutive patients with invasive breast cancer. Correlations between stiffness and clinicopathological factors, including tumor size, lymph node involvement, tumor subtypes, and stromal-related genes' expressions in primary breast tumor, were statistically examined. We identified that clinical tumor stiffness significantly correlated with lymph node involvement and invasive tumor size but not with hormonal receptor expressions, human epidermal growth factor receptor type 2 status, and ki67 labeling index by analyses of both categorical and continuous variables of stiffness. On multivariate analyses, axillary lymph node metastasis was an independent factor that influenced the stiffness of primary breast tumor. In the gene expression analyses, relatively hard tumors had a significantly high gene expression of lysyl oxidase compared with soft tumors. Our study showed a close relationship between primary tumor stiffness by elastographic evaluation and lymph node involvement in clinical breast cancer. Further investigations on tumor-related tissue stiffness are required.