Sky Cone

Week 2: Neural Network Basics

Section 2: Python and Vectorization

1. Video: Vectorization

What is vectorization?



https://github.com/liusongdu/coursera/blob/main/course1.ipynb

SIMD: single instruction multiple data

True or false. Vectorization cannot be done without a GPU.

1. True
2. False

2. Video: More Vectorization Examples

Neural network programming guideline

Whenever possible, avoid explicit for-loops.


Vectors and matrix valued functions

Say you need to apply the exponential operation on every element of a matrix/vector.
\( v=\left[\begin{array}{c}
v_1 \\
\vdots \\
v_n
\end{array}\right] \) --> 





Logistic regression derivatives




Implementing Logistic Regression

3. Video: Vectorizing Logistic Regression

Vectorizing Logistic Regression

\( z^{(1)}=w^T x^{(1)}+b \)	\( z^{(2)}=w^T x^{(2)}+b \)	\( z^{(3)}=w^T x^{(3)}+b \)
\( a^{(1)}=\sigma\left(z^{(1)}\right) \)	\( a^{(2)}=\sigma\left(z^{(2)}\right) \)	\( a^{(3)}=\sigma\left(z^{(3)}\right) \)

What are the dimensions of matrix X in this video?

1. (m, 1)
2. (n_x​, m)
3. (m, )
4. (m, n_x​)

4. Video: Vectorizing Logistic Regression's Gradient Output

Vectorizing Logistic Regression



Implementing Logistic Regression





How do you compute the derivative of b in one line of code in Python numpy?

1. m(np.sum(dZ))
2. 1 - m(np.sum(dZ))
3. 1 / m*(np.sum(dZ))
4. 1 * m(np.sum(dZ))

5. Video: Broadcasting in Python

Broadcasting example

Calories from Carbs, Proteins, Fats in 100g of different foods:


GitHub: ipynb




General Principle



Matlab/Octave: bsxfun

Which of the following numpy line of code would sum the values in a matrix A vertically?

1. A.sum(axis = 1)
2. A.sum(axis)
3. A.sum(axis = 0)

References

https://numpy.org/doc/stable/reference/generated/numpy.reshape.html

19.2 Broadcasting

6. Video: A Note on Python/Numpy Vectors

Jupyter notebook:
https://github.com/liusongdu/coursera/blob/9b135a78310736deb5adb6bf2c7491e89502a6a6/course3.ipynb

Python/numpy vectors



What kind of array has dimensions in this format: (10, ) ?

1. An identity array
2. A rank 0 array
3. A rank 1 array

7. Video: Quick tour of Jupyter/iPython Notebooks

-
-

8. Video: Explanation of Logistic Regression Cost Function (Optional)

-