Objectives and outcomes
Upon completion of the course, students master the basics of neural networks. They identify problems that can be solved with this approach and produce solutions. They are familiar with existing applications of neural networks. Students distinguish various types of neural networks, know their advantages and disadvantages and have a basic knowledge of the applications of neural networks in intelligent systems.
Lectures
A neuron model and network architecture. A single neuron. Transfer function. Multi-input neuron. Layers of neurons. Multi-layer neural networks. Perceptron. The Hamming network. Feedforward layer. Recurrent layer. Hopfield network. The perceptron training rule. Signal and weight vector spaces. Linear vector spaces. Linear independence. Inner product space. Norm and orthogonalisation. Gram-Schmidt orthogonalization. Vector expansion. Reciprocal basis vectors. Linear transformations for neural networks. Supervised Hebbian training. Linear associator. Hebb’s rule. Performance optimisation. The steepest descent. Conjugate gradient. Minimum mean square error algorithm. Convergence analysis. Adaptive filtering. Adaptive noise cancellation. Backpropagation. A multi-layer perceptron. Pattern classification. Function approximation. Backpropagation algorithm. Chain rule. Dynamic networks. Dynamic backpropagation. Real-time recurrent training. Backpropagation through time. Associative learning. Unsupervised Hebbian rule. Simple recognition networks. Competitive networks. Hamming network. Competitive training. Competitive layers in biology. Self-organising property maps. Radial basis function network. Function approximation. Pattern classification. Clustering. Non-linear optimisation. Grossberg network. An illusion. Normalising vision. Basic nonlinear model. A two-tier competitive network. Selection of transfer function. Kohonen learning rule. Hopfield network. Lyapunov function and amplification effect.
Practical classes
Using ready-made libraries and software solutions for training neural networks. An illustration of different types of neural networks that were covered in lectures. Neural network testing. Applications in function approximation, probability estimation, shape and pattern recognition, clustering, prediction.