Neural network construction and selection in nonlinear modeling

Isabelle Rivals (1) , Léon Personnaz (1)

1 ESA - Equipe de Statistique Appliquée (UMRS 1158)

In this paper, we study how statistical tools which are commonly used independently can advantageously be exploited together in order to improve neural network estimation and selection in nonlinear static modeling. The tools we consider are the analysis of the numerical conditioning of the neural network candidates, statistical hypothesis tests, and cross validation. We present and analyze each of these tools in order to justify at what stage of a construction and selection procedure they can be most useful. On the basis of this analysis, we then propose a novel and systematic construction and selection procedure for neural modeling. We finally illustrate its efficiency through large scale simulations experiments and real world modeling problems.

Keywords: Growing and pruning procedures; nonlinear regression; neural networks; least squares estimation; linear Taylor expansion; ill-conditioning detection; input selection; model selection; leave-one-out cross validation; statistical hypothesis tests.; statistical hypothesis tests

Domains: Neural and Evolutionary Computing [cs.NE]; Machine Learning [stat.ML]

Isabelle Rivals, Léon Personnaz. Neural network construction and selection in nonlinear modeling. IEEE Transactions on Neural Networks, 2003, 14 (4), pp.804-819. ⟨hal-00797670⟩