Neural Networks and Learning Theory

Students will get insight into different network architectures and their principles of operation. Notions like artificial intelligence and automatic learning will be made precise during the course. A central issue is the understanding of mathematical ideas underlying different network learning algorithms. This includes both positive solutions of problems and knowledge about limits of the approaches studied.

Contents

Some central network architectures are treated. These architectures differ in the way they manipulate input data, the way they perform learning tasks and the analysis of corresponding algorithms by mathematical means. More precisely, the following types of networks are covered:

  • General aspects of architecures, in particular feedforward nets, recurrent nets
  • Perceptron network, perceptron learning algorithm
  • Backpropagation algorithm
  • Radial basis function networks
  • Support Vector Machines
  • Learning theory and Vapnik-Chervonenkis dimension
  • Self-organizing networks
  • Hopfield networks

Special emphasis will be given to the mathematical analysis of algorithms.
This will make it necessary to study some basic facts of optimization and probability theory.

For students of the study programmes:

  • M.Sc. / Angewandte Mathematik (research-oriented profile) / Prüfungsordnung 2008
  • M.Sc. / Angewandte Mathematik (research-oriented profile) / Prüfungsordnung 2019
    Study programme Applied Mathematics M. Sc.: Compulsory elective module in "Mathematics-Enhancement"
     
  • M.Sc. / Cyber Security (research-oriented profile) / Prüfungsordnung 2017
    Study programme Cyber Security M.Sc.: Compulsory elective module in complex "Computer Science"
     
  • M.Sc. / Informatik (research-oriented profile) / Prüfungsordnung 2008 - 2. SÄ 2017
    Study programme Computer Science M. Sc.: Compulsory elective module in complex "Foundations of Computer Science" (level 400)
     
  • M.Sc. / Informations- und Medientechnik (research-oriented profile) / Prüfungsordnung 2017
    Study programme Information and Media Technology M.Sc.: compulsory elective module in „Fundamental Methods"
     
  • B.Sc. / Mathematik (research-oriented profile) / Prüfungsordnung 2019
  • B.Sc. / Wirtschaftsmathematik (research-oriented profile) / Prüfungsordnung 2019
  • Abschluss im Ausland / Cyber Security / keine Prüfungsordnung

If there is no need that the module is taught in English, alternatively the german version 12450 "Neuronal Netze und Lerntheorie" may be offered instead.

Literature

  • E. Alpaydin: Maschinelles Lernen, Oldenbourg Verlag München, 2008
  • M. Anthony, N.Biggs: Computational Learning Theory, Cambridge University Press 1997
  • N. Christiani, J. Shawe-Taylor: An Introduction to Support Vector Machines and kernel-based Learning Methods, Cambridge Univ. Press,  2003
  • A.C.C Coolen, R. Kühn, P. Sollich: Theory of Neural Information Processing Systems, Oxford University Press 2005
  • P. Fischer: Algorithmisches Lernen, Teubner 1999
  • P. Flach: Machine Learning: The Art and Science of Algorithms that Make Sense of Data, Cambridge University Press 2012
  • F. M. Ham, I. Kostanic: Principles of Neurocomputing for Science & Engineering, McGraw Hill 2001
  • S. Haykin: Neural Networks, Prentice Hall, 1999
  • R. Rojas: Theorie der neuronalen Netze, Springer 1996
  • S. Shalev-Shwartz, S. Ben-David: Understanding Machine Learning, Cambridge University Press 2014.