Quantum theory allows for randomness generation in a device-independent setting, where no detailed description of the experimental device is required. Here we derive a general upper bound on the amount of randomness that can be certified in such a setting. Our bound applies to any black-box scenario, thus covering a wide range of scenarios from partially characterized to completely uncharacterized devices. Specifically, we prove that the number of random bits that can be certified is limited by the number of different input states that enter the measurement device. We show explicitly that our bound is tight in the simplest cases. More generally, our paper indicates that the prospects of certifying a large amount of randomness by using high-dimensional (or even continuous variable) systems will be extremely challenging in practice.