Covalent organic frameworks (COFs) are an emerging class of highly porous crystalline organic polymers comprised entirely of organic linkers connected by strong covalent bonds. Due to their excellent physicochemical properties (e.g., ordered structure, porosity, and stability), COFs are considered ideal materials for developing state-of-the-art separation membranes. In fact, significant advances have been made in the last six years regarding the fabrication and functionalization of COF membranes. In particular, COFs have been utilized to obtain thin-film, composite, and mixed matrix membranes that could achieve effective rejection (mostly above 80%) of organic dyes and model organic foulants (e.g., humic acid). COF-based membranes, especially those prepared by embedding into polyamide thin-films, obtained adequate rejection of salts in desalination applications. However, the claims of ordered structure and separation mechanisms remain unclear and debatable. In this perspective, we analyze critically the design and exploitation of COFs for membrane fabrication and their performance in water treatment applications. In addition, technological challenges associated with COF properties, fabrication methods, and treatment efficacy are highlighted to redirect future research efforts in realizing highly selective separation membranes for scale-up and industrial applications.