The ubiquitous large, complex and high dimensional datasets in computer vision and machine learning generates the problem of subspace clustering, which aims to partition the data into several low dimensional subspaces. By utilizing relatively limited labeled data and sufficient unlabeled data, the semi-supervised subspace clustering is more effective, practical and become more popular. In this work, we present a new regularity combing the labels and the affinity to ensure the coherence of the affinity between data points from the same subspace as well as the discrimination of cluster labels for data points from different subspaces. We combine it with the manifold smoothing term of the existing methods and the Gaussian fields and harmonic functions method to give a new unified optimization framework for semi-supervised subspace clustering. Analysis shows the proposed model fully combines the affinity and the labels to guide each other so that both are discriminative between clusters and coherent within clusters. Extensive experiments show that our method outperforms the existing state-of-the-art methods, thus suggests that the property of discriminative between clusters and coherent within clusters of our method is advantageous to semi-supervised subspace clustering.