Response to the comments by J. Larsen and L. K. Hansen for : Rivals I. & Personnaz L. (2000), Construction of confidence intervals for neural networks based on least squares estimation (Neural Networks 13)

Abstract : We answer several comments made by Hansen and Larsen (2001) about our paper (Rivals & Personnaz, 2000). In this paper, we dealt with the construction of confidence intervals (CIs) for neural networks based on least squares (LS) estimation and using the linear Taylor expansion of the network output. We also suggested a method for the detection of the possible overfitting of a trained neural network, and an estimate of its leave-one-out (LOO) score that does not necessitate additional trainings. Finally, we showed that the frequentist approach we adopt compares favourably with other analytic approaches such as the conceptually very different Bayesian approach.