Graphene is an attractive photoconductive material for optical detection due to its broad absorption spectrum and ultrashort response time. However, it remains a great challenge to achieve high responsivity in graphene detectors because of graphene's weak optical absorption (only 2.3% in the monolayer graphene sheet) and short photocarrier lifetime (<1 ps). Here we show that metallic antenna structures can be designed to simultaneously improve both light absorption and photocarrier collection in graphene detectors. The coupled antennas concentrate free space light into the nanoscale deep-subwavelength antenna gaps, where the graphene light interaction is greatly enhanced as a result of the ultrahigh electric field intensity inside the gap. Meanwhile, the metallic antennas are designed to serve as electrodes that collect the generated photocarriers very efficiently. We also elucidate the mechanism of photoconductive gain in the graphene detectors and demonstrate mid-infrared (mid-IR) antenna-assisted graphene detectors at room temperature with more than 200 times enhancement of responsivity (∼0.4 V/W at λ0 = 4.45 μm) compared to devices without antennas (<2 mV/W).