Optical cavity cooling of mechanical resonators has recently become a research frontier where cooling of the vibrational motion of the resonators has been realized via photo-thermal force [1] and subsequently via radiation pressure [2–4]. One of the ultimate goals is reaching the vibrational ground state allowing quantum mechanical states of vibration - and the field offers a potential for hybrid quantum devices, and optical engineering of the phonon degrees of freedom. We present a new cavity cooling mechanism in semiconductors due to an optical excitation of carriers above the bandgap and the stress that these carriers introduce in the crystal lattice. The new method proves very power efficient and could potentially help in enabling ground state cooling of vibrational modes of semiconductor resonators. A laser cooled narrow-band phonon bath may open up a new avenue for photonics.