A major objective of pork producers is to reduce production cost. Feeding may account for over 75% of pork production costs. Thus, selecting pigs for feed efficiency (FE) traits is a priority in pig breeding programs. While in the Americas, pigs are typically fed high-input diets, based on corn and soybean meal (CS); in Western Europe, pigs are commonly fed diets based on wheat and barley with high amounts of added protein-rich coproducts (WB), e.g., from milling and seed-oil industries. These two feeding scenarios provided a realistic setting for investigating a specific type of genotype by environment interaction; thus, we investigated the genotype by feed interaction (GxF). In the presence of a GxF, different feed compositions should be considered when selecting for FE. This study aimed to 1) verify the presence of a GxF for FE and growth performance traits in different growth phases (starter, grower, and finisher) of 3-way crossbred growing-finishing pigs fed either a CS (547 boars and 558 gilts) or WB (567 boars and 558 gilts) diet; and 2) to assess and compare the expected responses to direct selection under the 2 diets and the expected correlated responses for one diet to indirect selection under the other diet. We found that GxF did not interfere in the ranking of genotypes under both diets for growth, protein deposition, feed intake, energy intake, or feed conversion rate. Therefore, for these traits, we recommend changing the diet of growing-finishing pigs from high-input feed (i.e., CS) to feed with less valuable ingredients, as WB, to reduce production costs and the environmental impact, regardless of which diet is used in selection. We found that GxF interfered in the ranking of genotypes and caused heterogeneity of genetic variance under both diets for lipid deposition (LD), residual energy intake (REI), and residual feed intake (RFI). Thus, selecting pigs under a diet different from the diet used for growing-finishing performance could compromise the LD in all growth phases, compromise the REI and RFI during the starter phase, and severely compromise the REI during the grower phase. In particular, when pigs are required to consume a WB diet for growing-finishing performance, pigs should be selected for FE under the same diet. Breeding pigs for FE under lower-input diets should be considered, because FE traits will become more important and lower-input diets will become more widespread in the near future.