Amplitude modulation (AM) may be an important factor for the perceived annoyance of wind turbine noise (WTN). Two AM types, typically referred to as “normal AM” (NAM) and “other AM” (OAM), characterize WTN AM, OAM corresponding to having intermittent periods with larger AM depth in lower frequency regions than NAM. The extent to which AM depth, frequency, and type affect WTN annoyance remains uncertain. Moreover, the temporal variations of WTN AM have often not been considered. Here, realistic stimuli accounting for such temporal variations were synthesized such that AM depth, frequency, and type, while determined from real on-site recordings, could be varied systematically. Listening tests with both original and synthesized stimuli showed that a reduction in mean AM depth across the spectrum led to a significant decrease in annoyance. When the spectrotemporal characteristics of the original far-field stimuli and the temporal AM variations were taken into account, the effect of AM frequency remained limited and the presence of intermittent OAM periods did not affect annoyance. These findings suggest that, at a given overall level, the AM depth of NAM periods is the most crucial AM parameter for WTN annoyance.