We evaluated the ablation thresholds of optical materials by using hard X-ray free electron laser. A 1-µm-focused beam with 10-keV of photon energy from SPring-8 Angstrom Compact free electron LAser (SACLA) was irradiated onto silicon and SiO2 substrates, as well as the platinum and rhodium thin films on these substrates, which are widely used for optical materials such as X-ray mirrors. We designed and installed a dedicated experimental chamber for the irradiation experiments. For the silicon substrate irradiated at a high fluence, we observed strong mechanical cracking at the surface and a deep ablation hole with a straight side wall. We confirmed that the ablation thresholds of uncoated silicon and SiO2 substrates agree with the melting doses of these materials, while those of the substrates under the metal coating layer are significantly reduced. The ablation thresholds obtained here are useful criteria in designing optics for hard X-ray free electron lasers.