Wiki
AI
Neural Networks
Convolutional
Interpretation

Interpretation

Activation Maximisation

  • Synthesise an ideal image for a class
    • Maximise 1-hot output
    • Maximise SoftMax

am

am-process

Regulariser

  • Fit to natural image statistics
  • Prone to high frequency noise
    • Minimise
  • Total variation
    • x∗x^* is the best solution to minimise loss
x∗=argminx∈RH×W×Cl(ϕ(x),ϕ0)x^*=\text{argmin}_{x\in \mathbb R^{H\times W\times C}}\mathcal l(\phi(x),\phi_0)
  • Won’t work x∗=argminx∈RH×W×Cl(ϕ(x),ϕ0)+λR(x)x^*=\text{argmin}_{x\in \mathbb R^{H\times W\times C}}\mathcal l(\phi(x),\phi_0)+\lambda\mathcal R(x)
  • Need a regulariser like above

am-regulariser

RVβ(f)=∫Ω((∂f∂u(u,v))2+(∂f∂v(u,v))2)β2du dv\mathcal R_{V^\beta}(f)=\int_\Omega\left(\left(\frac{\partial f}{\partial u}(u,v)\right)^2+\left(\frac{\partial f}{\partial v}(u,v)\right)^2\right)^{\frac \beta 2}du\space dvRVβ(x)=∑i,j((xi,j+1−xij)2+(xi+1,j−xij)2)β2\mathcal R_{V^\beta}(x)=\sum_{i,j}\left(\left(x_{i,j+1}-x_{ij}\right)^2+\left(x_{i+1,j}-x_{ij}\right)^2\right)^{\frac \beta 2}
  • Beta
    • Degree of smoothing
Inception LayerMax Pooling