A fast, power-efficient electro-optical vector-by-matrix multiplier (VMM) architecture is presented. Careful design of an electrical unit supporting high-speed data transfer enables this architecture to overcome bottlenecks encountered by previous VMM architectures. Based on the proposed architecture, we present an electro-optical digital signal processing (DSP) coprocessor that can achieve a significant speedup of 2-3 orders of magnitude over existing DSP technologies and execute more than 16 teraflops. We show that it is feasible to implement the system using off-the-shelf components, analyze the performance of the architecture with respect to primitive DSP operations, and detail the use of the new architecture for several DSP applications.