Asymptotic analysis of Bayesian generalization error with Newton diagram

Neural Netw. 2010 Jan;23(1):35-43. doi: 10.1016/j.neunet.2009.07.029. Epub 2009 Aug 7.

Abstract

Statistical learning machines that have singularities in the parameter space, such as hidden Markov models, Bayesian networks, and neural networks, are widely used in the field of information engineering. Singularities in the parameter space determine the accuracy of estimation in the Bayesian scenario. The Newton diagram in algebraic geometry is recognized as an effective method by which to investigate a singularity. The present paper proposes a new technique to plug the diagram in the Bayesian analysis. The proposed technique allows the generalization error to be clarified and provides a foundation for an efficient model selection. We apply the proposed technique to mixtures of binomial distributions.

MeSH terms

  • Algorithms
  • Artificial Intelligence*
  • Bayes Theorem*
  • Computer Simulation
  • Electronic Data Processing
  • Generalization, Psychological*
  • Humans
  • Information Storage and Retrieval