Optimizing the antecedent part of neuro-fuzzy system is investigated in a number of documents. Current approaches typically suffer from high computational complexity or lack of ability to extract knowledge from a given set of training data. In this paper, we introduce a novel incremental training algorithm for the class of neuro-fuzzy systems that are structured based on local linear classifiers. Linear discriminant analysis is utilized to transform the data into a space in which linear discriminancy of training samples is maximized. The neuro-fuzzy classifier is built in the transformed space, starting from the simplest form. In addition, rule consequent parameters are optimized using a local least square approach.