AI: Deep Learning 01

• Basic words
  • Data
    • Model’s input
    • Data augmentation is important for big data
    • Make Batch before getting into model
  • Model
    • Architecture such as LeNet, AlexNet, VGG, ResNet, etc.
    • Composed w/ layers such as Convolution Layer, Pooling, etc
    • Need to tune weights to get best features
  • Prediction / Logit
    • Predicted value for each class

      [0.15, 0.2, 0.6, 0.05] or [0.0, 0.0, 0.0, 1.0]

    • Larger value -> more correct answer
  • Loss / Cost
    • Loss function: Get difference (loss/cost) between predicted value and actual value
    • There’s a lot of Loss Function such as Cross Entropy
    • Learning in order to minimize Loss
  • Optimization
    • Minimize Loss
    • Does not apply optimized value but value w/ learning rate
  • Result
    • From predicted values, get max value using argmax
  • Layer
    • Input layer - Hidden Layers - Ouput layer
    • VGG16 : 16 layers
    • Not necessarily deeper the better
      • May cause over-fitting
      • Slower
      • Heavy
  • Weight / Filter / Kernel / Variable / Bias

  y = W(eight)x + b(ias)

  • Convolution: Get feature

  • Pooling: Zip feature to exagarate it

  • Activation Function
    • Sigmoid

      σ(x)=1/1+e^-x

    • Leaky ReLU

      max(0.1x, x)

    • tanh

      tanh(x)

    • ReLU

      max(0, x)

    • Maxout

      max(ω^T₁x+b₁,ω^T₂x+b₂)

    • ELU

      x >= 0 ? x : α(e^x-1)

  • Softmax
    • Make logit scores into probabilities

    <img src=https://cdn-images-1.medium.com/max/906/1*670CdxchunD-yAuUWdI7Bw.png>

  • Learning rate
    • One of hyper parameter that human need to set in order to make learn more effective

    <img src=https://www.jeremyjordan.me/content/images/2018/02/Screen-Shot-2018-02-24-at-11.47.09-AM.png>

  • Batch size
    • Set number of data
  • Epoch / Step
    • Inspect data again and again
    • Count for repeated inspection
  • Train / Validation / Test
    • Divide data for each case.
  • Label / Ground Truth
    • Correct answer
• CNN Architecture
  • Feature Extraction: Convolution + ReLU + Max Pooling
    • Convolution layer &rightarrow; Get feature w/ filter
    • Pooling layer &rightarrow; Retrieve important feature and zip it
    • Activation function &rightarrow; Delete unimportant feature
  • Classification: Fully Connected Layer &rightarrow; Decide answer w/ weight and bias