Development of small molecules targeting the PD-L1/PD-1 interface is advancing both in industry and academia, but only a few have reached early-stage clinical trials. Here, we take a closer look at the general druggability of PD-L1 using in silico hot spot mapping and nuclear magnetic resonance (NMR)-based characterization. We found that the conformational elasticity of the PD-L1 surface strongly influences the formation of hot spots. We deconstructed several generations of known inhibitors into fragments and examined their binding properties using differential scanning fluorimetry (DSF) and protein-based nuclear magnetic resonance (NMR). These biophysical analyses showed that not all fragments bind to the PD-L1 ectodomain despite having the biphenyl scaffold. Although most of the binding fragments induced PD-L1 oligomerization, two compounds, TAH35 and TAH36, retain the monomeric state of proteins upon binding. Additionally, the presence of the entire ectodomain did not affect the binding of the hit compounds and dimerization of PD-L1. The data demonstrated here provide important information on the PD-L1 druggability and the structure-activity relationship of the biphenyl core moiety and therefore may aid in the design of novel inhibitors and focused fragment libraries for PD-L1.