We demonstrate the use of focused ion beam milling to machine optical structures directly into the core of microstructured optical fibers. The particular fiber used was exposed-core microstructured optical fiber, which allowed direct access to the optically guiding core. Two different designs of Fabry-Perot cavity were fabricated and optically characterized. The first cavity was formed by completely removing a section of the fiber core, while the second cavity consisted of a shallow slot milled into the core, leaving the majority of the core intact. This work highlights the possibility of machining complex optical devices directly onto the core of microstructured optical fibers using focused ion beam milling for applications including environmental, chemical, and biological sensing.