Cancer initiation, progression, and invasion occur in a complex and dynamic microenvironment which depends on the hosts and sites where tumors develop. Tumors arising in mucosal tissues may progress in an inflammatory context linked to local viral and/or bacterial infections. At the opposite, tumors developing in immunoprivileged sites are protected from microorganisms and grow in an immunosuppressive environment. In the present review, we summarize and present our recent data on the influence of infectious context and immune cell infiltration organization in human Non-Small Cell Lung Cancers (NSCLC) progression. We show that stimulation of tumor cells by TLR for viral ssRNA, such as TLR7/8, or bacteria, such as TLR4, promotes cell survival and induces chemoresistance. On the opposite, stimulation by TLR3, receptor for double-stranded viral RNA, decreases tumor cell viability and induces chemosensitivity in some lung tumor cell lines. Since fresh lung tumor cells exhibit a gene expression profile characteristic of TLR-stimulated lung tumor cell lines, we suspect that viral and bacterial influence may not only act on the host immune system but also directly on tumor growth and sensitivity to chemotherapy. The stroma of NSCLC contains tertiary lymphoid structures (or Tumor-induced Bronchus-Associated Lymphoid Tissues (Ti-BALT)) with mature DC, follicular DC, and T and B cells. Two subsets of immature DC, Langerhans cells (LC) and interstitial DC (intDC), were detected in the tumor nests and the stroma, respectively. Here, we show that the densities of the three DC subsets, mature DC, LC, and intDC, are highly predictive of disease-specific survival in a series of 74 early-stage NSCLC patients. We hypothesize that the mature DC may derive from local activation and migration of the immature DC--and especially LC which contact the tumor cells--to the tertiary lymphoid structures, after sampling and processing of the tumor antigens. In view of the prominent role of DC in the immune response, we suggest that the microenvironment of early-stage NSCLC may allow the in situ activation of the adaptive response. Finally, we find that the eyes or brain of mice with growing B cell lymphoma are infiltrated with T cells and that the cytokines produced ex vivo by the tumoral tissues have an impaired Th1 cytokine profile. Our work illustrates that the host and external tumor microenvironments are multifaceted and strongly influence tumor progression and anti-tumor immune responses.