Subjects attending full-time special education (SE) often have multifactorial background for their cognitive impairment, and brain MRI may show nonspecific changes. As voxel-based morphometry reveals regional volume differences, we applied this method to 119 subjects with cognitive impairments and familial need for full-time SE—graded into three levels from specific disorders of cognitive processes (level 1) to intellectual disability (IQ <70; level 3)—and to 43 age-matched controls attending mainstream education (level 0). Subjects in SE groups had smaller global brain white matter (WM), cerebrospinal fluid, and total brain volume than controls. Compared with controls, subjects with intellectual disabilities in SE level 3 showed greater regional gray matter volumes bilaterally in the ventral and dorsal anterior cingulate cortex and smaller regional gray matter volumes in the left thalamus and cerebellar hemisphere. Further, they had greater WM volume in the left frontoparietal region and smaller WM volumes in the posterior limbs of the internal capsules. Subjects in SE level 1 and 2 groups showed the same tendency, but the results were nonsignificant. In conclusion, compared with controls, subjects with intellectual disabilities showed in voxel-based morphometry analysis several regional brain alterations.