Abstract: A novel pruning algorithm, which can keep the performance of the network while its neurons are removed one by one, is proposed. In each step, a hidden unit is chosen to be deleted according to one of two proposed rules, then a linear least square problem is solved to adjust part of the remaining weights in order that the performance of the reduced network is as close as possible to the original one. Compared with the existing pruning algorithms, the proposed method may lead to networks with smaller size. The simulation results of finding the functional relationship between GDP(gross domestic product) and GE(gross export), GI(gross import) show the effectiveness of the proposed method.