Naturally-occurring membranes in the xylem tissue of gymnosperm sapwood enable its use as an abundantly-available material to construct filters, with potential to facilitate access to safe drinking water in resource-constrained settings. However, the material's behavior as a filter is poorly understood, and challenges such as short shelf life have not been addressed. Here, we characterize the operational attributes of xylem filters and show that the material exhibits a highly non-linear dependence of flow resistance on thickness upon drying, and a tendency for self-blocking. We develop guidelines for the design and fabrication of xylem filters, demonstrate gravity-operated filters with shelf life >2 years, and show that the filters can provide >3 log removal of E. coli, MS-2 phage, and rotavirus from synthetic test waters and coliform bacteria from contaminated spring, tap, and ground waters. Through interviews and workshops in India, we use a user-centric approach to design a prototype filtration device with daily- to weekly-replaceable xylem filters, and uncover indicators of social acceptance of xylem as a natural water filter. Our work enhances the understanding of xylem as a filtration material, and opens opportunities for engineering a diverse range of low-cost, biodegradable xylem-based filtration products on a global scale.