The desorption of neutral H atoms from graphite with femtosecond XUV pulses is reported. The velocity distribution of the atoms peaks at extremely low kinetic energies. A DFT-based electron scattering calculation traces this distribution to desorption out of specific adsorption sites on graphite, and identifies the highest vibrational state in the adsorbate potential as a major source for the slow atoms. It is evident that multiple electron scattering processes are required for this desorption. A direct electronic excitation of a repulsive hydrogen-carbon bond seems not to be important.