Impedance control is capable of further flexibly adjusting the driving-point impedance of series elastic actuators (SEAs) in addition to impedance reduction by the elastic element. This enhances safety and compliance during interaction between humans and robots, in comparison with rigid robots under impedance control or SEAs under position control. In this study, we propose an impedance controller for SEA systems that is developed based on the singular perturbation (SP) theory and time-delay estimation (TDE) technique. The SP theory allows for alleviating the burden of the requirement for states to be measured. The TDE technique is effective in compensating for system dynamics and uncertainties involved in system behaviors with minute computation loads. Employing both a simulation study and experimental study, we demonstrate the efficacy of the proposed control created from the combination of the SP theory and TDE technique. The effect of the proposed impedance control on reducing the driving-point impedance of interacting SEAs is examined.