**Description**

**Syllabus**

**Beginners**

**Unit-1: Introduction, Motivation and History of Neural Network**- Neural Networks
- Applications
- Classification of Data
- Anomaly Detection
- Speech Recognition
- Audio Generation
- Time Series Analysis
- Spell Checking
- Character Recognition
- Machine Translation
- Image Processing

- General Structure
- Perception
- Steps involved in a Neural Network
- Feedforward
- Backpropagation

- Why do we need Backpropagation?
- Basic Flow of Neural Networks
- The 100-step rule
- Simple Application Examples
- The Classical Way
- The Way of Learning
- A Brief History of Neural Networks
- The Beginning
- Golden Age
- Long Silence and Slow Reconstruction
- Renaissance

**Unit-2: Biological Neural Networks**- Biological Overview
- The Vertebrate Nervous System
- Peripheral and Central Nervous System
- The Cerebrum in Responsible for Abstract Thinking Processes
- The Cerebellum Controls and Coordinates Motor Functions
- The Diencephalon Controls Fundamental Physiological Processes
- The Brainstem Connects the Brain with the Spinal Cord and Controls Reflexes
- Neurons are Information Processing Cells
- Components of a Neuron
- Synapses Weight the Individual Parts of Information
- Neurotransmitters
- Dendrites Collect all Parts of Information
- In the Soma, the Weighted Information is Accumulated
- The Axon Transfers Outgoing Pulses
- Electrochemical Processes in the Neuron and Its Components
- Neurons Maintain Electrical Membrane Potential
- Membrane Potential
- The Neuron is Activated by Changes in the Membrane Potential
- Threshold and Resting State
- Initiation of Action Potential Over Time
- In the Axon, a Pulse is Conducted in a Saltatory Way
- Receptor Cells are Modified Neurons
- There are Different Receptors Cells for Various Types of Perceptions
- Information is Processed on Every Level of the Nervous System
- The Information Processing is Entirely Decentralized
- An Outline of Common Light Sensing Organs
- Compound Eyes and Pinhole Eyes Only Provide High Temporal or Spatial Resolution
- Single Lens Eyes Combine the Advantages of the other Two Eye Types, but They are More Complex
- The Retina does not Only Receive Information but is also Responsible for Information Processing
- Steps of Information Processing
- Horizontal and Amacrine Cells
- The Amount of Neurons in Living Organisms at Different Stages of Development
- Transition to Technical Neurons: Neural Networks are a Caricature of Biology
- Steps of Information Processing
- Through Radical Simplification Briefly Summarize the Conclusions Relevant for the Technical Part
- Our Brief Summary Corresponds Exactly with the Few Elements of Biological Neural Networks we want to Take Over into the Technical Approximation
- Exercises

**Unit-3: Components of Artificial Neural Networks**- The Concept of Time in Neural Networks
- Components of Neural Networks
- Data Processing of a Neuron
- Connections Carry Information That is Processed by Neurons
- The Propagation Converts Vector Inputs to Scalar Network Inputs
- The Activation is the “Switching Status” of a Neuron
- Neurons Get Activated If The Network Input Exceeds Their Threshold Value
- The Activation Function Determines the Activation of a Neuron Dependent on Network Input and Threshold Value
- Common Activation Functions
- An Output Function May Be Used to Process the Activation once again
- Learning Strategies Adjust a Network to Fit Our Needs
- Network Topologies
- Feed-Forward Networks Consist of Layers and Connections Towards Each Following Layer
- Feed Forward Network
- Shortcut Connections Skip Layers
- Direct Recurrences start and end at the same Neuron
- Indirect Recurrences Can Influence Their Starting Neuron only by making Detours
- Lateral Recurrences Connect Neurons Within One Layer
- Completely Linked Networks Allow any Possible Connection
- The Bias Neuron is a Technical Trick to Consider Threshold Values as Connection Weights
- Representing Neurons
- Take care of the order in which Neuron Activations are Calculated
- Synchronous Activation
- Asynchronous Activation
- Random Order
- Random Permutation
- Topological Order
- Fixed Orders of Activation During Implementation
- Communication with The Outside World: Input and Output of Data in and from Neural Networks
- Exercises

**Unit-4: Fundamentals of learning and training samples**- Learning and Training
- Neural Networks is their Capability
- Different paradigms of learning
- About Neuron Functions
- Learning Algorithm
- Training Set
- Unsupervised Learning
- Supervised Learning methods
- Reinforcement Learning
- Offline or Online Learning
- Training Patterns and teaching Input
- Error Vector
- Using Training Samples
- Divisions of training samples
- Training Sample Lesson
- Order of Pattern Representation
- Learning curve and error measurement
- Specific Error
- Root mean Square and Total Error
- Stop Learning
- Gradient Optimization Procedures
- Gradient Dimension
- Errors during a gradient descent
- Gradient Descent
- Gradient Descents against suboptimal minima
- Flat Plateaus on the error surface may cause training slowness
- Even if good minima are reached, they may be left afterwards
- Steep canyons in the error surface may cause Oscillations
- Exemplary problems allow for testing self-coded learning strategies
- Boolean Functions
- The parity Function
- The 2-spiral problem
- The Checkerboard problem
- Samples for the Checkerboard problem
- The identity function
- Other exemplary problems
- The Hebbian Learning
- Original Rule
- Generalized Form
- Exercises

**Unit-5: The Perceptron, Backpropagation & its Variants****Unit-6: Radial Basis Functions(RBF)**- Introduction
- Components & Structure of an RBF Network
- Center of an RBF Neuron
- RBF Neuron
- RBF Output Neuron
- RBF Network
- RBF Network with Input Neurons
- Individual one or two dimensional
- Information processing of an RBF network
- Different Gaussian Bells
- Information Processing in RBF neurons
- Gaussian bells in two-dimensional space
- Gaussian bell
- Analytical Thoughts
- Equations of Weights
- Generalization on Several Outputs
- Computational effort and accuracy
- Combinations of the equation system
- Fixed Selection & Conditional fixed selection
- 2-D Input Space
- Uneven coverage of a 2-D Input Space
- Growing RBF Networks
- Neurons are added to places with large error values
- Limiting the number of neurons
- Less Important neurons are deleted
- Comparing RBF networks & Multilayer Perceptrons

**Unit-7: Recurrent Perceptron-like networks**- Recurrent neural networks
- Jordan Networks
- Elman Networks
- Training recurrent networks
- Unfolding in time
- Teacher Forcing
- Recurrent backpropagation
- Training with evolution

**Unit-8: Hopfield Networks**- Hopfield Networks
- Hopfield networks are inspired by particles in a magnetic field
- In a Hopfield network, all neurons influence each other symmetrically
- State of a Hopfield Network
- Input and output of a Hopfield network
- Significance of weights
- A neuron changes its state
- The weight matrix is generated directly out of the training patterns
- Learning rule for Hopfield networks
- Auto association and traditional application
- Pattern Recognition
- Hetero association and analogies to neural data storage
- Hopfield Network
- Generating the heteroassociative matrix
- Heteroassociative Matrix
- Stabilizing the Heteroassociations
- The weight matrix is generated directly out of the training patterns
- The biological motivation of hetero association
- Continuous Hopfield networks

**Unit-9: Learning Vector Quantization**- Introduction of Learning Vector Quantization
- About Quantization
- LVQ divides the input space into separate areas
- Quantization of a two-dimensional input space
- Using codebook vectors: the nearest one is the winner
- Adjusting codebook vectors
- The procedure of learning
- LVQ learning procedure
- Learning process

**Unit-10: Self-Organizing Feature Maps**- Unsupervised Learning
- Structure of a self-organizing map
- One-dimensional grid
- Self-organizing map
- Topology
- SOMs always activate the neuron
- Training
- Adapting the centres
- SOM learning rule
- Topology function defines
- Introduction of common distance and topology functions
- Decrease Monotonically
- Gaussian bell, cone function, cylinder function and the Mexican hat function
- Learning direction
- Our topology function
- The learning rate
- Topological defects
- The behaviour of a SOM
- End states of one-dimensional (left column) and two-dimensional (right column)
- Topological defect in two-dimensional SOM
- Adjust Resolution of certain areas in a SOM
- Training of a SOM
- SOMs can be used to determine centres for RBF neurons
- Neural Gas
- A figure filled by a SOM
- Multi-SOM
- Multi-Neural Gas
- Growing Neural Gases

**Unit-11: Adaptive Resonance Theory**- Introduction of Adaptive Resonance Theory
- Task and structure of an Adaptive Resonance Theory
- Resonance takes place by activities being tossed and turned
- Top-down & Bottom-up Learning
- Pattern input & top-down learning
- Resonance and bottom-up learning

**Appendix-A: Excursus Cluster Analysis and Regional & Online Learnable Fields**- Introduction
- Metric
- A.1 k-means clustering allocates data to a predefined number of clusters
- A.2 k-nearest neighbouring looks for the k nearest neighbours of each data point
- A.3 ?-nearest neighbouring looks for neighbours within the radius ? for each data point
- A.3 ?-nearest neighbouring looks for neighbours within the radius ? for each data point
- A.4 The silhouette coefficient determines how accurate a given clustering is
- A.5 Regional and online learnable fields are a neural clustering strategy
- A.5.1 ROLFs try to cover data with neurons
- A.4 The silhouette coefficient determines how accurate a given clustering is

- A.5.2 A ROLF learns unsupervised by presenting training samples online

- A.5.1 ROLFs try to cover data with neurons
- ROLF neuron and Perceptive surface
- Structure of a ROLF neuron
- Accepting neuron
- Both positions and radii are adapted throughout the learning
- The radius multiplier allows neurons to be able not only to shrink
- As required, new neurons are generated
- Evaluating a ROLF
- ROLF
- Comparison with popular clustering methods
- Initializing radii, learning rates and multiplier is not trivial

**Appendix-B: Excursus: Neural Networks used for Prediction**- Introduction
- About time series
- One-step-ahead prediction
- Moving Average Procedure
- Two-step-ahead prediction
- Recursive two-step-ahead prediction
- Direct two-step-ahead prediction
- Additional optimization approaches for prediction
- Changing temporal parameters
- Heterogeneous prediction
- Remarks on the prediction of share prices

**Appendix-C: Excursus: Reinforcement Learning**- Introduction
- Reinforcement Learning
- System Structure
- Grid World
- Agent and Environment
- In the Grid world
- Environment
- States, situations and actions
- Reward and return
- Closed Loop Policy
- Exploitation vs. Exploration
- Learning process
- Rewarding strategies
- Avoidance Strategy
- The state-value function
- Policy evaluation
- Policy Improvement
- Monte Carlo method
- Temporal difference learning
- The action-value function
- Q learning

**Intermediate**

**Advanced**

**Professional**