lab05_2_Logistic_Regression

▷ Cost Function

• the cost function to fit the parameters(θ)
• Given the training set how to we chose/fit θ?
  • $h_θ(x) = y$ then Cost = 0
  • $cost(h_θ,(x),y) = -ylog(h_θ(x)) - (1-y)log(1-h_θ(x))$

def loss_fn(hypothesis, labels):
    cost = -tf.reduce_mean(labels * tf.log(hypothesis) + (1 - labels) * tf.log(1 - hypothesis))
    return cost

• A convex logistic regression cost function
• (y = 1)
  • if the result becomes smaller than 1, the cost increases
  • because it becomes closer to 0 than the desired prediction value of 1
• (y = 0)
  • if the result becomes bigger than 0, the cost increases
  • becouase it becomes closer to 1 than the desired prediction value of 0

cost = -tf.reduce_mean(labels * tf.log(hypothesis) + (1 - labels) * tf.log(1 - hypothesis))

▷ Optimization

• How to minimize the cost function
• Repeat { $θ_j := θ_j - α\frac {∂}{∂θ_j}J(θ)$ }
  • learning rate : $α$
  • cost : $J(θ)$
  • gradient : $\frac {∂}{∂θ_j}$

def grad(hypothesis, labels):
    with tf.GradientTape() as tape:
        loss_value = loss_fn(hypothesis, labels)
    return tape.gradient(loss_value, [W, b])
optimizer = tf.GradientDescentOptimizer(learning_rate = 0.01)
optimizer.apply_gradients(grads_and_vars=zip(grads,[W,b]))