Due to the combined effect of anisotropic interactions and activity, Janus swimmers are capable to self-assemble in a wide variety of structures, many more than their equilibrium counterpart. This might lead to the development of novel active materials capable of performing tasks without any central control. Their potential application in designing such materials endows trying to understand the fundamental mechanism in which these swimmers self-assemble. In the present work, we study a quasi-two-dimensional semidilute suspensions of two classes of amphiphilic spherical swimmers whose direction of motion can be tuned: either swimmers propelling in the direction of the hydrophobic patch or swimmers propelling in the opposite direction (toward the hydrophilic side). In both systems we have systematically tuned swimmers' hydrophobic strength and signature and observed that the anisotropic interactions, characterized by the angular attractive potential and its interaction range, in competition with the active stress, pointing toward or against the attractive patch gives rise to a rich aggregation phenomenology.