Wiki
AI
Neural Networks
Learning
Competitive Learning
  • Only single output neuron fires
  1. Set of homogeneous neurons with some randomly distributed synaptic weights
    • Respond differently to given set of input patterns
  2. Limit imposed on strength of each neuron
  3. Mechanism to allow neurons to compete for right to respond to a given subset of inputs
    • Only one output neuron active at a time
    • Or only one neuron per group
    • Winner-takes-all neuron

  • Lateral inhibition
    • Neurons inhibit other neurons
  • Winning neuron must have highest induced local field for given input pattern
    • Winning neuron is squashed to 1
    • Others are clamped to 0
yk={1if vk>vj for all j,j≠k0otherwisey_k= \begin{cases} 1 & \text{if } v_k > v_j \text{ for all } j,j\neq k \\ 0 & \text{otherwise} \end{cases}
  • Neuron has fixed amount of weight spread amongst input synapses
    • Sums to 1
  • Learn by shifting weights from inactive to active input nodes
    • Each input node relinquishes some proportion of weight
    • Distributed amongst active nodes
Δwkj={η(xj−wkj)if neuron k wins the competition0if neuron k loses the competition\Delta w_{kj}= \begin{cases} \eta(x_j-w_{kj}) & \text{if neuron $k$ wins the competition}\\ 0 & \text{if neuron $k$ loses the competition} \end{cases}
  • Individual neurons learn to specialise on ensembles of similar patterns
    • Feature detectors
BoltzmannCredit Assignment Problem