Although many fish are soniferous, few of their sounds have been identified, making passive acoustic monitoring (PAM) ineffective. To start addressing this issue, a portable 6-hydrophone array combined with a video camera was assembled to catalog fish sounds in the wild. Sounds are detected automatically in the acoustic recordings and localized in three dimensions using time-difference of arrivals and linearized inversion. Localizations are then combined with the video to identify the species producing the sounds. Uncertainty analyses show that fish are localized near the array with uncertainties < 50 cm. The proposed system was deployed off Cape Cod, MA and used to identify sounds produced by tautog (Tautoga onitis), demonstrating that the methodology can be used to build up a catalog of fish sounds that could be used for PAM and fisheries management.