Cationic liposomes and polymers are both important candidates for use as non-viral gene vectors. However, both of them have special shortcomings and application limits. This work is devoted to the combination of advantages of liposomes and polymers. The ring-opening polymerization strategy was used for the preparation of amphiphilic polymers from cyclen-based cationic small lipids. The non-hydrophobic polymer and the corresponding lipids were also prepared for performing structure-activity relationship studies. Gel electrophoresis results reveal that both the lipopolymers and liposomes could effectively condense DNA into nanoparticles and protect DNA from degradation. Compared to polymers, the DNA binding ability of liposomes is more affected by hydrophobic tails. Under the same dosage, the synthetic polymers have stronger DNA binding ability than the liposomes. In vitro transfection experiments show that the polymers could give better transfection efficiency, which was much higher than those of the corresponding liposomes and non-hydrophobic polymer. The oleyl moiety is suitable for lipidic vectors, but things were different for polymers. Under optimized conditions, up to 14.2 times higher transfection efficiency than that for 25 kDa bPEI could be obtained. More importantly, the lipopolymers showed much better serum tolerance, which was further confirmed by protein adsorption, gel electrophoresis, flow cytometry, and CLSM assays. The results indicate that ring-opening polymerization is a promising strategy for the enhancement of the gene delivery efficiency and biocompatibility of cationic lipids.