Self-structuring hidden control neural models

The majority of neural models for pattern recognition have fixed architecture during training. A typical consequence is non-optimal and often too large networks. In this paper we propose a Self-structuring Hidden Control (SHC) neural model for pattern recognition, which establishes a near optimal architecture during training. We typically achieve a significant network architecture reduction in terms of the number of hidden Processing Elements (PE). The SHC model combines self-structuring architecture generation with non-linear prediction and hidden Markov modelling. The paper presents a theorem for self-structuring neural models stating that these models are universal approximators and thus relevant for real-world pattern recognition. Using SHC models containing as few as five hidden PES each for an isolated word recognition task resulted in a recognition rate of 98.4%. SHC models can furthermore be applied to continuous speech recognition.