Solid-state electrolytes (SSEs) are highly desirable for next-generation energy storage and conversion devices. Herein, composite SSEs based on layered double hydroxides (LDHs)/metal-borohydrides (M-BH4 with M = Li, Na, Mg), with a remarkable increase in ionic conductivities, are proposed. The two-dimensional (2D) host layers of LDH, with rich surface hydroxyl groups, maximize the interface interacting area with M-BH4. The consequent defects of the interfacial M-BH4 enable a lower activation energy for boosting ion transfer. As a result, the surfaces of the LDHs serve as an advanced track for improving the ionic mobility of M-BH4. The resultant MgAl-LDH/LiBH4 exhibited a five order of magnitude increase in the room temperature ionic conductivity, reaching values of up to 0.31 mS cm−1, which is superior to most of the previously reported LiBH4-based SSEs. This work provides a new perspective for understanding how the surface effect promotes ionic conductivity of composite SSEs and an avenue towards the realization of low-temperature advanced SSEs.