Self Organizing Maps (SOM)

AI Summary

An AI-generated summary of this video lecture.

The lecture introduces Self Organizing Maps (SOM), also known as Self-Organizing Feature Maps (SOFM), which are a type of artificial neural network (ANN) trained using unsupervised learning. The primary goal is to produce a low-dimensional, typically two-dimensional, discretized representation of the input space, effectively serving as a method for dimensionality reduction. The instructor emphasizes that SOMs differ from other neural networks by applying competitive learning instead of error-correction learning methods like backpropagation with gradient descent. The session defines competitive learning as a process where output neurons compete to be activated, resulting in a single 'winning neuron' or 'winner-takes-all' outcome. The lecture also touches upon the historical context, noting the method was introduced by Teuvo Kohonen in the 1980s and is sometimes called a Kohonen feature map or topology preserving map.

Chapters

1. 0:00 – 2:00 00:00-02:00

   The video begins with a slide titled 'Self Organizing Maps(SOM)'. The text defines SOM as a type of artificial neural network (ANN) trained using unsupervised learning to produce a low-dimensional representation of the input space. The instructor underlines 'unsupervised learning' and 'map' to highlight the core function. She explains that this process is a method for dimensionality reduction. A key distinction is made in the second bullet point: SOMs apply competitive learning as opposed to error-correction learning. The instructor underlines 'competitive learning' and 'error-correction learning'. The third bullet point defines competitive learning, stating that output neurons compete amongst themselves to be activated, with the result that only one is activated at any one time. This activated neuron is called a winner-takes-all neuron or simply the winning neuron.

2. 2:00 – 5:00 02:00-05:00

   The instructor elaborates on how competition is implemented, mentioning lateral inhibition connections (negative feedback paths) between neurons. She draws a diagram showing input units connecting to a grid of output units, labeling it as a 2D map. The text explains that neighborhood constraints are imposed on the output units. The instructor notes that in SOM, we update not only the weights of the winning neuron but also the weights of its neighbors in the neighborhood around it. The size of the neighborhood generally decreases slowly with each iteration. The slide mentions that SOM was introduced by Finnish professor Teuvo Kohonen in the 1980s. The diagram shows a simple SOM with 2 inputs ($x_1$ and $x_2$) and 49 outputs. The instructor writes '1982' and 'Hope' on the side, likely referring to the year and the concept of hope or topology.

3. 5:00 – 7:58 05:00-07:58

   The final segment details the algorithm steps. A formula for the square of the Euclidean distance ($D_j$) is shown: $D_j = \sum_{i=1}^{n} \sum_{j=1}^{m} (x_i - w_{i,j})^2$. Step 3 is to find the winning unit index $j$ so that $D_j$ is minimum. Step 4 provides the weight update formula for units $j$ within a specific neighborhood: $w_{ij}( ext{new}) = w_{ij}( ext{old}) + \eta(x_i - w_{ij}( ext{old}))$. The learning rate is updated as $\eta(t+1) = 0.5 * \eta(t)$. The instructor then presents a worked example to construct KSOM to cluster four given vectors: $[0, 0, 1, 1]$, $[1, 0, 0, 0]$, $[0, 1, 1, 0]$, and $[0, 0, 0, 1]$. The number of clusters is two, and the learning rate is 0.5. She initializes weights randomly and calculates the Euclidean distance for the first input vector $X = [0, 0, 1, 1]$. The calculation shows $D_1 = 0.4$ and $D_2 = 2.04$. Since $D_1 < D_2$, cluster 1 is identified as the winning cluster.

The video provides a comprehensive overview of Self Organizing Maps, starting with their definition as unsupervised ANNs for dimensionality reduction. It clearly distinguishes SOMs from other networks by focusing on competitive learning and the winner-takes-all mechanism. The instructor explains the role of lateral inhibition and neighborhood constraints in organizing neurons. The lesson transitions into the mathematical algorithm, detailing the Euclidean distance calculation and weight update rules. Finally, a concrete numerical example demonstrates the application of these rules, guiding students through the process of clustering vectors and identifying winning neurons. This structured approach ensures a clear understanding of both the theory and practice of SOMs.