Learning Rules

AI Summary

An AI-generated summary of this video lecture.

The lecture provides a comprehensive overview of learning rules in Artificial Neural Networks (ANNs), defining them as mathematical logics used to update weights and biases to enhance network performance. The session covers the Hebbian Learning Rule, the Delta Learning Rule (Widrow-Hoff), and the Perceptron Learning Rule. A significant portion is dedicated to a detailed, step-by-step worked example demonstrating how to train a perceptron using the Hebbian rule for Bipolar AND logic, including weight initialization, net input calculation, and iterative weight updates until convergence.

Chapters

1. 0:00 – 2:00 00:00-02:00

   The video begins with a slide titled "Learning Rules," defining a learning rule or process as a method or mathematical logic. The text states that this rule improves the Artificial Neural Network's performance and applies the rule over the network. The speaker explains that learning rules update the weights and bias levels of a network when it simulates in a specific data environment. A "Point to Note" section appears, emphasizing that classification in a perceptron is done on the basis of the target output, which directs the classification process. The slide also mentions that the target output directs the classification and an example will be shown in the next slide. The instructor underlines "classification in a perceptron is done on the basis of the target output" to stress its importance.

2. 2:00 – 5:00 02:00-05:00

   The instructor transitions to discussing the mechanics of learning rules, writing "Learning Rule" and "How to update" on the screen to illustrate the concept of weight and bias adjustment. She writes "op execute" and draws arrows to show the flow from input to output. She mentions that while some rules work for certain problems, others like XOR fail with single-layer perceptrons. The lecture introduces the Hebbian Learning Rule, one of the oldest and simplest rules, introduced by Donald Hebb. The formula $\Delta w = \eta \cdot t \cdot x_i$ is presented to increase the weight of connection at every time step. She writes "Mistake -> Correct -> weight" to explain the feedback loop.

3. 5:00 – 10:00 05:00-10:00

   The Hebbian Learning Rule is detailed further. The slide describes it as a kind of feed-forward, unsupervised learning. The formula components are defined: $\Delta w$ is the weight updation, $\eta$ is the learning rate, $t$ is the target output, and $x_i$ is the input value. The instructor writes $w_{new} = w_{old} + \Delta w$ to show how the weight change is applied. She emphasizes that this rule is used to increase the weight of connection at every time step based on the correlation between input and target. The text highlights "one of the oldest and simplest" and "Donald Hebb". She also writes "Learning Rule" and "How to update" again to reinforce the concept.

4. 10:00 – 15:00 10:00-15:00

   The lecture shifts to the Delta Learning Rule, also known as the Widrow-Hoff Rule or Least Mean Square method. It is introduced by Bernard Widrow and Marcian Hoff to minimize the error over all training patterns. The formula for updating synaptic weights is given as $\Delta w = \eta (t - o) x_i$, where $t$ is the Target Value and $o$ is the Perceptron Output. The session then moves to a practical example: Bipolar AND Logic Using Perceptron and Hebb rule. A table is shown with inputs $x_1, x_2$ and Target $t$. Weights $w_1, w_2$ and bias $b$ are initialized to zero, and the learning rate is set to 1. The slide notes that the example uses bipolar inputs (1, -1) instead of (1, 0). The instructor highlights "Least Mean Square method" and "minimize the error over all training patterns".

5. 15:00 – 19:02 15:00-19:02

   The worked example proceeds with the first input pattern [1, 1, 1]. The net input is calculated as $y = b + x_1 w_1 + x_2 w_2 = 0 + 1(0) + 1(0) = 0$. Since the output $f(y) = 0$ does not match the target $t = 1$, weights are updated. The new weights become $W_1 = 1, W_2 = 1, b = 1$. For the second pattern [1, -1, -1], the net input is calculated as $y = 1 + 1(1) + (-1)(1) = 1$. The output $f(y) = 1$ does not match the target $t = -1$, so weights are updated again. The process continues for subsequent patterns until the final weights are determined as $W_1 = 1, W_2 = 1, b = -1$. The lecture briefly touches upon OR and XOR logics, noting that XOR inputs are not linearly separable. The instructor writes "Fail XOR" to indicate the limitation of single-layer perceptrons.

The lecture effectively bridges theoretical definitions with practical application. It starts by establishing what learning rules are—mathematical methods for weight adjustment—before diving into specific algorithms like Hebbian and Delta rules. The transition from the Hebbian formula to a concrete AND logic example allows students to see the abstract math in action. By walking through the initialization, calculation, and update steps, the video clarifies how a perceptron learns to classify data. The distinction between linearly separable problems (AND, OR) and non-separable ones (XOR) is also highlighted, providing context for why different rules or network architectures might be needed. The detailed calculation of the AND logic example serves as a crucial guide for students to understand the iterative nature of training neural networks. The instructor's handwritten notes and underlining help emphasize key concepts like "target output" and "Least Mean Square method".