Field-driven emission of electrons from metals is a technology that has had a large impact on science and society over the past century, for instance, in early and contemporary vacuum tube electronics, cathode ray displays, and high-power microwave and sub-millimeter electronics. In this Perspective article, we discuss the role of field-driven electron emission in the broader context of tunneling phenomena, and we discuss some aspects of the physics behind field emission. We focus on the use of ultrafast single-cycle transients of electromagnetic radiation in the terahertz (THz) frequency range to drive the electron emission process. With electric fields of such short duration, it becomes possible to generate electron bursts of sub-picosecond duration controllable on the sub-cycle time scale due to the large nonlinearity of the emission process. We review the recent literature on THz-driven field emission and then finally present selected experiments from our own laboratories to emphasize aspects such as design of the emitter structure with respect to optimal field enhancement properties, measurement of the electron bunch duration, visualization of the spatial emission pattern of emitted electrons, and material damage caused by the electrons.