Using filter-space techniques, we study the scale-to-scale transport of energy in a quasi-two-dimensional, weakly turbulent fluid flow averaged along the trajectories of fluid elements. We find that although the spatial mean of this Lagrangian-averaged flux is nearly unchanged from its Eulerian counterpart, the spatial structure of the scale-to-scale energy flux changes significantly. In particular, its features appear to correlate with the positions of Lagrangian coherent structures (LCS's). We show that the LCS's tend to lie at zeros of the scale-to-scale flux, and therefore that the LCS's separate regions that have qualitatively different dynamics. Since LCS's are also known to be impenetrable barriers to advection and mixing, we therefore find that the fluid on either side of an LCS is both kinematically and dynamically distinct. Our results extend the utility of LCS's by making clear the role they play in the flow dynamics in addition to the kinematics.