Simplification via aggregation for neural networks

Abstract

Aggregation is a useful technique for simplifying multicomponent systems. We illustrate this by sim plifying a class of neural network models and com paring the behaviors of the simplified (lumped) model and the original (base) model. Naturally, in order for aggregation to be applicable, certain constraints on the neural network models are imposed. The lumped model obtained is a vastly simplified model and is deterministic, even though the base model is proba bilistic. Nevertheless, by experimenting with the lumped model and using the information obtained in the form of force field plots, we are able to cor rectly predict many aspects of base model behavior. A related probabilistic lumped model, similar to the deterministic lumped model, is also examined.