Synaptic transistors that accommodate concurrent signal transmission and learning in a neural network are attracting enormous interest for neuromorphic sensory processing. To remove redundant sensory information while keeping important features, artificial synaptic transistors with non-linear conductance are desired to apply filter processing to sensory inputs. Here, we report the realization of non-linear synapses using a two-dimensional van der Waals (vdW) heterostructure (MoS2/h-BN/graphene) based float gate memory device, in which the semiconductor channel is tailored via a surface acceptor (ZnPc) for subthreshold operation. In addition to usual synaptic plasticity, the memory device exhibits highly non-linear conductance (rectification ratio >106), allowing bidirectional yet only negative/inhibitory current to pass through. We demonstrate that in a lateral coupling network, such a float gate memory device resembles the key lateral inhibition function of horizontal cells for the formation of an ON-center/OFF-surround receptive field. When combined with synaptic plasticity, the lateral inhibition weights are further tunable to enable adjustable edge enhancement for early visual processing. Our results here hopefully open a new scheme toward early sensory perception via lateral inhibitory synaptic transistors.