Metasurfaces have shown promising potentials in shaping optical wavefronts while remaining compact compared to bulky geometric optics devices. The design of meta-atoms, the fundamental building blocks of metasurfaces, typically relies on trial and error to achieve target electromagnetic responses. This process includes the characterization of an enormous amount of meta-atom designs with varying physical and geometric parameters, which demands huge computational resources. In this paper, a deep learning-based metasurface/meta-atom modeling approach is introduced to significantly reduce the characterization time while maintaining accuracy. Based on a convolutional neural network (CNN) structure, the proposed deep learning network is able to model meta-atoms with nearly freeform 2D patterns and different lattice sizes, material refractive indices and thicknesses. Moreover, the presented approach features the capability of predicting a meta-atom's wide spectrum response in the timescale of milliseconds, attractive for applications necessitating fast on-demand design and optimization of a meta-atom/metasurface.