We demonstrate optical fibers designed for the THz frequency range, fabricated in a low-loss polymer. The polymer fibers display a broadband loss of 0.4 dB/cm over the 0.1-1 THz range, with a minimum loss of 0.1 dB/cm in the region near 500 GHz. The fibers, based on endlessly single-mode design, have tailored dispersion and may be bent into sharp bends. Due to the confinement of the THz field in the core of the fibers they are ideal for stable guiding of THz light in confined environments, and may serve as a useful basis for a wealth of fiber-based photonic components in the THz range, particularly in spectroscopic applications where tight confinement of the THz field is required. We further demonstrate a new spectroscopic technique for ultrafast time-resolved THz time-domain spectroscopy which simultaneously acquires both reference and sample data. By using this scheme we show that the influence of fluctuations on the laser parameters during data acquisition can be minimized, and highly reproducible quantitative data can be recorded and extracted in a very efficient manner. This technique may become especially important in the high THz range, where phase noise becomes critical for the accuracy of a measurement.