Although an impact noise level is objectively evaluated the same according to current standards, a lightweight floor structure is often subjectively judged more annoying than a heavy homogeneous structure. The hypothesis of the present investigation is that the subjective judgment of impact noise is more annoying if the source position can be localized; lightweight structures have a more localized radiation than heavy structures. For the heavy structures the reverberant vibration field is dominant, therefore having a distributed radiation. A listening test is used to assess the subjective annoyance, using simulated binaural room impulse responses, with sources being a moving point source or a nonmoving surface source, and rooms being a room with a reverberation time of 0.5 s or an anechoic room. The paper concludes that no strong effect of the source localization on the annoyance can be found.