Microbial fuel cell (MFC) technology is a simple way to accelerate the treatment of the oily sludge which is a major problem affecting the quality of oil fields and surrounding environment while generating electricity. To investigate the oil removal and the characteristics of changes in the composition of bacteria, sediment microbial fuel cells (SMFCs) supplemented with oily sludge was constructed. The results showed that the degradation efficiency of total petroleum hydrocarbon (TPH) of SMFC treatment was 10.1 times higher than the common anaerobic degradation. In addition, the degradation rate of n-alkanes followed the order of high carbon number > low carbon number > medium carbon number. The odd-even alkane predominance (OEP) increased, indicating that a high contribution of even alkanes whose degradation predominates. The OUT number, Shannon index, AEC index, and Chao1 index of the sludge treated with SMFC (YN2) are greater than those of the original sludge (YN1), showing that the microbial diversity of sludge increased after SMFC treatment. After SMFC treatment the relative abundance of Chloroflexi, Bacteroidia and Pseudomonadales which are essential for the degradation of the organic matter and electricity production increased significantly in YN2. These results will play a crucial role in improving the performance of oily sludge MFC.