Silicon-based optical neural networks offer the prospect of high-performance computing on integrated photonic circuits. However, the scalability of on-chip optical depth networks is restricted by the limited energy and space resources. Here, we present a silicon-based photonic convolutional neural network (PCNN) combined with the kernel pruning, in which the optical convolutional computing core of PCNN is a tunable micro-ring weight bank. Our numerical simulation demonstrates the effect of weight mapping accuracy on PCNN performance and we find that the performance of PCNN decreases significantly when the weight mapping accuracy is less than 4.3 bits. Additionally, the experimental demonstration shows that the accuracy of the PCNN on the MNIST dataset has a slight loss compared to the original CNN when 93.75 % of the convolutional kernels are pruned. By making use of kernel pruning, the energy saved by a convolutional kernel removal is about 202.3 mW, and the overall energy saved has a linear relationship with the number of kernels removed. The methodology is scalable and provides a feasible solution for implementing faster and more energy-efficient large-scale optical convolutional neural networks on photonic integrated circuits.