We investigate clustering of particles at an initially flat fluid-fluid interface of surface tension γ under an external force f directed perpendicular to the interface. We employ analytical theory, numerical energy minimization (Surface Evolver) and computational fluid dynamics (the Lattice-Boltzmann method) to study the equilibrium deformation of the interface and structural mechanics of the clusters, in particular at the onset of instability. In the case of incompressible clusters, we find that the equilibrium 3D interface profiles are uniquely determined by the length scale γ/(fn0), where n0 is the particle surface number density, and a non-dimensional shape parameter f2Nn0/γ2. The scaling remains valid in the whole regime of forces f, i.e., even close to the stability limit fcrit. In the cases with an initial hexagonal arrangement of the particles, upon f approaching fcrit, our simulations additionally reveal the emergence of curvature-induced defects and 2D stress anisotropy. We develop stability diagrams in terms of f, N (we study 7 ≤ N ≤ 61), and the contact angle θp at the particles and identify three unstable regimes corresponding to (i) collective detachment of the whole cluster from the interface, (ii) ejection of individual particles, and (iii) both detachment and ejection. We also discuss possible metastable states. Altogether, our results may help in better understanding and controlling the particle interfacial instabilities with potential uses in synthesis of new materials, environmental sciences and microfluidics.