Prediction of prognosis in colorectal cancer is vital for the choice of therapeutic options. Histopathological factors remain paramount in this respect. Factors such as tumor size, histological type and subtype, presence of signet ring morphology and the degree of differentiation as well as the presence of lymphovascular invasion and lymph node involvement are well known factors that influence outcome. Our understanding of these factors has improved in the past few years with factors such as tumor budding, lymphocytic infiltration being recognized as important. Likewise the prognostic significance of resection margins, particularly circumferential margins has been appreciated in the last two decades. A number of molecular and genetic markers such as KRAS, BRAF and microsatellite instability are also important and correlate with histological features in some patients. This review summarizes our current understanding of the main histopathological factors that affect prognosis of colorectal cancer.