Wiki
AI
Neural Networks
Learning
Boltzmann
  • Stochastic
  • Recurrent structure
  • Binary operation (+/- 1)
  • Energy function
E=−12∑j∑kwkjxkxjE=-\frac 1 2 \sum_j\sum_k w_{kj}x_kx_j
  • j≠kj\neq k
  • No self-feedback
  • xx = neuron state
  • Neurons randomly flip from xx to −x-x
P(xk→−xk)=11+e−ΔEkTP(x_k \rightarrow-x_k)=\frac 1 {1+e^{\frac{-\Delta E_k}{T}}}
  • Energy change based on pseudo-temperature
    • System will reach thermal equilibrium
  • Delta E is the energy change resulting from the flip
  • Visible and hidden neurons
    • Visible act as interface between network and environment
    • Hidden always operate freely

Operation Modes

  • Clamped
    • Visible neurons are clamped onto specific states determined by environment
  • Free-running
    • All neurons able to operate freely
  • ρkj+\rho_{kj}^+ = Correlation between states while clamped
  • ρkj−\rho_{kj}^- = Correlation between states while free
  • Both exist between +/- 1
Δwkj=η(ρkj+−ρkj−), j≠k\Delta w_{kj}=\eta(\rho_{kj}^+-\rho_{kj}^-), \space j\neq k
Competitive Learning