Molecular dynamics simulations of all-atom and coarse-grained lipid bilayer models are increasingly used to obtain useful insights for understanding the structural dynamics of these assemblies. In this context, one crucial point concerns the comparison of the performance and accuracy of classical force fields (FFs), which sometimes remains elusive. To date, the assessments performed on different classical potentials are mostly based on the comparison with experimental observables, which typically regard average properties. However, local differences of the structure and dynamics, which are poorly captured by average measurements, can make a difference, but these are nontrivial to catch. Here, we propose an agnostic way to compare different FFs at different resolutions (atomistic, united-atom, and coarse-grained), by means of a high-dimensional similarity metrics built on the framework of Smooth Overlap of Atomic Position (SOAP). We compare and classify a set of 13 FFs, modeling 1-palmitoyl-2-oleoyl-sn-glycero-3-phosphocholine (POPC) bilayers. Our SOAP kernel-based metrics allows us to compare, discriminate, and correlate different FFs at different model resolutions in an unbiased, high-dimensional way. This also captures differences between FFs in modeling nonaverage events (originating from local transitions), for example, the liquid-to-gel phase transition in dipalmitoylphosphatidylcholine (DPPC) bilayers, for which our metrics allows us to identify nucleation centers for the phase transition, highlighting some intrinsic resolution limitations in implicit versus explicit solvent FFs.