Morphological control of multimetallic nanostructures is crucial for obtaining shape-dependent physical and chemical properties. Up to date, control of the shapes of multimetallic nanostructures has remained largely empirical. Multimetallic nanostructures have been produced mostly through seed-mediated growth. Understanding the role played by starting nanocrystal seeds can help in controlling the shape and in turn the plasmonic and catalytic properties of multimetallic nanostructures. In this work, we have studied the effect of the crystalline structure and shape of Au nanocrystal seeds on the morphology of the resultant bimetallic nanostructures. Single-crystalline Au nanorods, multiply twinned Au nanorods, and multiply twinned Au nanobipyramids were employed as the starting seeds. Both silver and palladium exhibit highly preferential growth on the side surfaces of the single-crystalline Au nanorods, giving rise to bimetallic cuboids, whereas they prefer to grow at the ends of the multiply twinned Au nanorods and nanobipyramids, giving rise to bimetallic nanorods. These results indicate that the morphology of the bimetallic nanostructures is highly dependent on the crystalline structure of the Au nanocrystal seeds. Our results will be useful for guiding the preparation of multimetallic nanostructures with desired shapes and therefore plasmonic properties for various plasmon-based applications.