Complex-valued shift-invariant canonical polyadic decomposition (CPD) under a spatial phase sparsity constraint (pcsCPD) shows excellent separation performance when applied to band-pass filtered complex-valued multi-subject fMRI data. However, some useful information may also be eliminated when using a band-pass filter to suppress unwanted noise. As such, we propose an alternating rank- R and rank-1 least squares optimization to relax the CPD model. Based upon this optimization method, we present a novel constrained CPD algorithm with temporal shift-invariance and spatial sparsity and orthonormality constraints. More specifically, four steps are conducted until convergence for each iteration of the proposed algorithm: 1) use rank- R least-squares fit under spatial phase sparsity constraint to update shared spatial maps after phase de-ambiguity; 2) use orthonormality constraint to minimize the cross-talk between shared spatial maps; 3) update the aggregating mixing matrix using rank- R least-squares fit; 4) utilize shift-invariant rank-1 least-squares on a series of rank-1 matrices reconstructed by each column of the aggregating mixing matrix to update shared time courses, and subject-specific time delays and intensities. The experimental results of simulated and actual complex-valued fMRI data show that the proposed algorithm improves the estimates for task-related sensorimotor and auditory networks, compared to pcsCPD and tensorial spatial ICA. The proposed alternating rank- R and rank-1 least squares optimization is also flexible to improve CPD-related algorithm using alternating least squares.