We analytically calculate the optical emission spectrum of nanolasers and nano-LEDs based on a model of many incoherently pumped two-level emitters in a cavity. At low pump rates, we find two peaks in the spectrum for large coupling strengths and numbers of emitters. We interpret the double-peaked spectrum as a signature of collective Rabi splitting, and discuss the difference between the splitting of the spectrum and the existence of two eigenmodes. We show that an LED will never exhibit a split spectrum, even though it can have distinct eigenmodes. For systems where the splitting is possible, we show that the two peaks merge into a single one when the pump rate is increased. Finally, we compute the linewidth of the systems, and discuss the influence of inter-emitter correlations on the lineshape.