Biochemical reaction networks perform a variety of signal processing functions, one of which is computing the integrals of signal values. This is often used in integral feedback control, where it enables a system's output to respond to changing inputs, but to then return exactly back to some pre-determined setpoint value afterward. To gain a deeper understanding of how biochemical networks are able to both integrate signals and perform integral feedback control, we investigated these abilities for several simple reaction networks. We found imperfect overlap between these categories, with some networks able to perform both tasks, some able to perform integration but not integral feedback control, and some the other way around. Nevertheless, networks that could either integrate or perform integral feedback control shared key elements. In particular, they included a chemical species that was neutrally stable in the open loop system (no feedback), meaning that this species does not have a unique stable steady-state concentration. Neutral stability could arise from zeroth order decay reactions, binding to a partner that was produced at a constant rate (which occurs in antithetic control), or through a long chain of covalent cycles. Mathematically, it arose from rate equations for the reaction network that were underdetermined when evaluated at steady-state.