We investigate whether field homogeneity of a magnetic assembly can be optimized by varying the remanence of its constituting magnetic segments. We specifically study this hypothesis for a Halbach cylinder using a numerical model, MagTense. We consider a Halbach cylinder consisting of six layers of three concentric rings, each ring made from 16 segments. We show that ideally, the homogeneity can reach close to 1 ppm for a finite magnet. We then proceed to consider a real world set of magnet segments, i.e. non-ideal magnets with a variation in their remanence. This reduces the field homogeneity to about 1000 ppm when considering a Gaussian perturbation of the remanence with a standard deviation of 1%. However, we also show that the reduction in homogeneity may be countered by organizing the magnet pieces found through optimization, which is possible if each magnet segment is well characterized experimentally. We note that the presented method is applicable to any case where homogeneity of the field is important. The results we present are considered for the specific case of nuclear magnetic resonance for concretenes.