Iron oxide nanoparticles are used in various industrial fields, as a tool in biomedicine as well as in food colorants, and can therefore reach human metabolism via oral uptake or injection. However, their effects on the human body, especially the liver as one of the first target organs is still under elucidation. Here, we studied the influence of different representative iron oxide materials on xenobiotic metabolism of HepaRG cells. These included four iron oxide nanoparticles, one commercially available yellow food pigment (E172), and non-particulate ionic control FeSO4. The nanoparticles had different chemical and crystalline structures and differed in size and shape and were used at a concentration of 50 µg Fe/mL. We found that various CYP enzymes were downregulated by some but not all iron oxide nanoparticles, with the Fe3O4-particle, both γ-Fe2O3-particles, and FeSO4 exhibiting the strongest effects, the yellow food pigment E172 showing a minor effect and an α-Fe2O3 nanoparticle leading to almost no inhibition of phase I machinery. The downregulation was seen at the mRNA, protein expression, and activity levels. Thereby, no dependency on the size or chemical structure was found. This underlines the difficulty of the grouping of nanomaterials regarding their physiological impact, suggesting that every iron oxide nanoparticle species needs to be evaluated in a case-by-case approach.