Bone is a biological composite material consisting of two main components: collagen and mineral. Collagen is the most abundant protein in vertebrates, which makes it of high clinical and scientific interest. In this paper, we compare the composition and structure of cortical bone demineralized using several protocols: ethylene-diamine-tetraacetic acid (EDTA), formic acid (CH2O2), hydrochloric acid (HCl), and HCl/EDTA mixture. The efficiencies of these four agents were investigated by assessing the remaining mineral quantities and collagen integrity with various experimental techniques. Raman spectroscopy results show that the bone demineralized by the CH2O2 agent has highest collagen quality parameter. The HCl/EDTA mixture removes the most mineral, but it affects the collagen secondary structure as amide II bands are shifted as observed by Fourier transform infrared spectroscopy. Thermogravimetric analysis reveals that HCl and EDTA are most effective in removing the mineral with bulk measurements. In summary, we conclude that HCl best demineralizes bone, leaving the well-preserved collagen structure in the shortest time. These findings guide on the best demineralization protocol to obtain high-quality collagen from bone for clinical and scientific applications.