Contrastive learning has recently attracted plenty of attention in deep graph clustering due to its promising performance. However, complicated data augmentations and time-consuming graph convolutional operations undermine the efficiency of these methods. To solve this problem, we propose a simple contrastive graph clustering (SCGC) algorithm to improve the existing methods from the perspectives of network architecture, data augmentation, and objective function. As to the architecture, our network includes two main parts, that is, preprocessing and network backbone. A simple low-pass denoising operation conducts neighbor information aggregation as an independent preprocessing, and only two multilayer perceptrons (MLPs) are included as the backbone. For data augmentation, instead of introducing complex operations over graphs, we construct two augmented views of the same vertex by designing parameter unshared Siamese encoders and perturbing the node embeddings directly. Finally, as to the objective function, to further improve the clustering performance, a novel cross-view structural consistency objective function is designed to enhance the discriminative capability of the learned network. Extensive experimental results on seven benchmark datasets validate our proposed algorithm's effectiveness and superiority. Significantly, our algorithm outperforms the recent contrastive deep clustering competitors with at least seven times speedup on average. The code of SCGC is released at SCGC. Besides, we share a collection of deep graph clustering, including papers, codes, and datasets at ADGC.