Investigating epidemiology of metastatic colon and rectal cancer is challenging, because cancer registries seldom record metastatic sites. We used a population based approach to assess metastatic spread in colon and rectal cancers. 49,096 patients with colorectal cancer were identified from the nationwide Swedish Cancer Registry. Metastatic sites were identified from the National Patient Register and Cause of Death Register. Rectal cancer more frequently metastasized into thoracic organs (OR = 2.4) and the nervous system (1.5) and less frequently within the peritoneum (0.3). Mucinous and signet ring adenocarcinomas more frequently metastasized within the peritoneum compared with generic adenocarcinoma (3.8 [colon]/3.2 [rectum]), and less frequently into the liver (0.5/0.6). Lung metastases occurred frequently together with nervous system metastases, whereas peritoneal metastases were often listed with ovarian and pleural metastases. Thoracic metastases are almost as common as liver metastases in rectal cancer patients with a low stage at diagnosis. In colorectal cancer patients with solitary metastases the survival differed between 5 and 19 months depending on T or N stage. Metastatic patterns differ notably between colon and rectal cancers. This knowledge should help clinicians to identify patients in need for extra surveillance and gives insight to further studies on the mechanisms of metastasis.