The silicon photomultiplier (SiPM) is a recently developed photosensor used in particle physics, e.g., for detection of minimum ionizing particles and/or Cherenkov radiation. Its performance is comparable to that of photomultiplier tubes, but with advantages in terms of reduced volume and magnetic field insensitivity. In the present study, the performance of a gamma ray detector made of an yttrium aluminum perovskite scintillation crystal and a SiPM-based readout is assessed for use in time of flight neutron spectroscopy. Measurements performed at the ISIS pulsed neutron source demonstrate the feasibility of gamma-detection based on the new device.