The human skeletal form underlies our ability to walk on two legs, but unlike standing height, the genetic basis of limb lengths and skeletal proportions is less well understood. Here we applied a deep learning model to 31,221 whole body dual-energy X-ray absorptiometry (DXA) images from the UK Biobank (UKB) to extract 23 different image-derived phenotypes (IDPs) that include all long bone lengths as well as hip and shoulder width, which we analyzed while controlling for height. All skeletal proportions are highly heritable (∼40-50%), and genome-wide association studies (GWAS) of these traits identified 179 independent loci, of which 102 loci were not associated with height. These loci are enriched in genes regulating skeletal development as well as associated with rare human skeletal diseases and abnormal mouse skeletal phenotypes. Genetic correlation and genomic structural equation modeling indicated that limb proportions exhibited strong genetic sharing but were genetically independent of width and torso proportions. Phenotypic and polygenic risk score analyses identified specific associations between osteoarthritis (OA) of the hip and knee, the leading causes of adult disability in the United States, and skeletal proportions of the corresponding regions. We also found genomic evidence of evolutionary change in arm-to-leg and hip-width proportions in humans consistent with striking anatomical changes in these skeletal proportions in the hominin fossil record. In contrast to cardiovascular, auto-immune, metabolic, and other categories of traits, loci associated with these skeletal proportions are significantly enriched in human accelerated regions (HARs), and regulatory elements of genes differentially expressed through development between humans and the great apes. Taken together, our work validates the use of deep learning models on DXA images to identify novel and specific genetic variants affecting the human skeletal form and ties a major evolutionary facet of human anatomical change to pathogenesis.