The thermal boundary conductance of Al/SiO2, Al/Si, Au/SiO2, and Au/Si are measured by a femtosecond laser transient thermoreflectance technique. The distinct differences of the interfacial thermal conductance between these samples are observed. For the same metal film, the thermal boundary conductance between metal and substrate decreases with the thermal conductivity of the substrate. The measured results are explained with the phonon diffusion mismatch model by introducing a phonon transmission coefficient across the interface.