Sky Cone

Week 2: Neural Network Basics

Section 1: Logistic Regression as a Neural Network

1. Video: Binary Classification

Binary Classification



dimension = 64 * 64 * 3 = 12288
n = n_x = 12288

Notation

2. Video: Logistic Regression

Logistic Regression


-


What are the parameters of logistic regression?

1. W, an n_x​ dimensional vector, and b, a real number.
2. W and b, both n_x​ dimensional vectors.
3. W and b, both real numbers.
4. W, an identity vector, and b, a real number.

3. Video: Logistic Regression Cost Function

Logistic Regression cost function



What is the difference between the cost function and the loss function for logistic regression?

1. The loss function computes the error for a single training example; the cost function is the average of the loss functions of the entire training set.
2. The cost function computes the error for a single training example; the loss function is the average of the cost functions of the entire training set.
3. They are different names for the same function.

4. Video: Gradient Descent

Gradient Descent

Recap:
\( \hat{y}=\sigma\left(w^T x+b\right) \), \( \sigma(z)=\frac{1}{1+e^{-z}} \)

cost function:
\( J(w, b)=\frac{1}{m} \sum_{i=1}^m \mathcal{L}\left(\hat{y}^{(i)}, y^{(i)}\right)=-\frac{1}{m} \sum_{i=1}^m y^{(i)} \log \hat{y}^{(i)}+\left(1-y^{(i)}\right) \log \left(1-\hat{y}^{(i)}\right) \)

Want to find \( w \), \( b \) that minimize \( J(w, b) \)




convex function







True or false. A convex function always has multiple local optima.

1. True
2. False

5. Video: Derivatives

Intuition about derivatives



On a straight line, the function's derivative...

1. changes as values on axis increase/decrease.
2. doesn't change.

nudge 移动
infinitesimal 无限小量

6. Video: More Derivative Examples

Intuition about derivatives
-


More derivative examples

7. Video: Computation Graph

Computation Graph



One step of ________ propagation on a computation graph yields derivative of final output variable.

1. Backward
2. Forward

8. Video: Derivatives with a Computation Graph

Compute deriratives





In this class, what does the coding convention dvar represent?

1. The derivative of a final output variable with respect to various intermediate quantities.
2. The derivative of any variable used in the code.
3. The derivative of input variables with respect to various intermediate quantities.

9. Video: Logistic Regression Gradient Descent

Logistic regression recap



Logistic regression deriratives
-
\( z=w_1 x_1+w_2 x_2+b \) ----> \( a=\sigma(z) \) ----> \( \mathcal{L}(\mathrm{a}, y) \)



In this video, what is the simplified formula for the derivative of the losswith respect to z?  

1. a (1 - y)
2. a / (1-a)
3. a - y

10. Video: Gradient Descent on m Examples

Logistic regression on m examples





In the for loop depicted in the video, why is there only one dw variable (i.e. no i superscripts in the for loop)?

1. The value of dw in the code is cumulative.
2. Only the derivative of one value is relevant.
3. Only one derivative is being computed.

11. Reading: Derivation of DL/dz (Optional)

Derivation of \( \frac{d L}{d z} \)​

If you're curious, you can find the derivation for \( \frac{d L}{d z}=a-y \) in this Discourse post "Derivation of DL/dz"

Remember that you do not need to know calculus in order to complete this course or the other courses in this specialization. The derivation is just for those who are curious about how this is derived.