Due to its ultrahigh electron transmissivity in a wide electron energy range, molecular impermeability, high electrical conductivity and excellent mechanical stiffness, suspended graphene membranes appear to be a nearly ideal window material for in situ (in vivo) environmental electron microscopy of nano- and mesoscopic objects (including bio-medical samples) immersed in liquids and/or in dense gaseous media. In this paper, taking advantage of a small modification of the graphene transfer protocol onto metallic and SiN supporting orifices, reusable environmental cells with exchangeable graphene windows have been designed. Using colloidal gold nanoparticles (50 nm) dispersed in water as model objects for scanning electron microscopy in liquids as proof of concept, different conditions for imaging through the graphene membrane were tested. Limiting factors for electron microscopy in liquids, such as electron beam induced water radiolysis and damage of the graphene membrane at high electron doses, are discussed.