Rice straw was hydrolyzed into a mixture of sugars using diluted H2SO4. During hydrolysis, a variety of inhibitors was also produced, including acetic acid, furfural, hydroxymethylfurfural, and lignin degradation products (several aromatic and phenolic compounds). To reduce the toxic compounds concentration in the hydrolyzate and to improve the xylitol yield and volumetric productivity, rice straw hemicellulosic hydrolyzate was treated with activated charcoal under different pH values, stirring rates, contact times, and temperatures, employing a 24 full-factorial design. Fermentative assays were conducted with treated hydrolyzates containing 90 g/L xylose. The results indicated that temperature, pH, and stirring rate strongly influenced the hydrolyzate treatment, temperature and pH interfering with all of the responses analyzed (removal of color and lignin degradation products, xylitol yield factor, and volumetric productivity). The combination of pH 2.0, 150 rpm, 45 °C, and 60 min was considered an optimal condition, providing significant removal rates of color (48.9%) and lignin degradation products (25.8%), as well as a xylitol production of 66 g/L, a volumetric productivity of 0.57 g/L·h, and a yield factor of 0.72 g/g.