Amyloid proteins are associated with a broad spectrum of neurodegenerative diseases. However, it remains a grand challenge to extract molecular structure information from intracellular amyloid proteins in their native cellular environment. To address this challenge, we developed a computational chemical microscope integrating 3D mid-infrared photothermal imaging with fluorescence imaging, termed Fluorescence-guided Bond-Selective Intensity Diffraction Tomography (FBS-IDT). Based on a low-cost and simple optical design, FBS-IDT enables chemical-specific volumetric imaging and 3D site-specific mid-IR fingerprint spectroscopic analysis of tau fibrils, an important type of amyloid protein aggregates, in their intracellular environment. Label-free volumetric chemical imaging of human cells with/without seeded tau fibrils is demonstrated to show the potential correlation between lipid accumulation and tau aggregate formation. Depth-resolved mid-infrared fingerprint spectroscopy is performed to reveal the protein secondary structure of the intracellular tau fibrils. 3D visualization of the \b{eta}-sheet for tau fibril structure is achieved.