Nanoparticles (NPs) that form by self-assembly of amphiphilic poly(N-(2-hydroxypropyl)-methacrylamide) (pHPMA) copolymers bearing cholesterol side groups are potential drug carriers for solid tumor treatment. Here, we investigate their behavior in solutions of human serum albumin (HSA) in phosphate buffered saline. Mixed solutions of NPs, from polymer conjugates with or without the anticancer drug doxorubicin (Dox) bound to them, and HSA at concentrations up to the physiological value are characterized by synchrotron small-angle X-ray scattering and isothermal titration calorimetry. When Dox is absent, a small amount of HSA molecules bind to the cholesterol groups that form the core of the NPs by diffusing through the loose pHPMA shell or get caught in meshes formed by the pHPMA chains. These interactions are strongly hindered by the presence of Dox, which is distributed in the pHPMA shell, meaning that the delivery of Dox by the NPs in the human body is not affected by the presence of HSA.