Hi,

This week you'll learn about Automatic Differentiation Part 1: Understanding the Math.

Image

Deep Learning can generate fashion photography, movie storyboards, viral tweets, and so much more. Sometimes when we look at this exponential progress, it might feel overwhelming. But it is equally important to realize that all of this started with the simple technique of teaching machines how to automatically calculate the error between predicted and actual values. 

A conscious differential engine generated by Stable Diffusion v2 (source: Stable Diffusion v2).

The big picture: To achieve this (progressively decrease error) by tuning a Neural Network's weights and biases (trainable parameters), we need to derive their gradients with respect to the objective functions. Computing the gradients by hand (as we did in our introductory calculus class) seems tiring.

With the present Deep Learning Frameworks (TensorFlow, JAX, PyTorch, etc.), one simply defines the forward pass to obtain the output from the objective function. The gradient computation happens under the hood. This process is called Automatic Differentiation (AD).

How it works: AD works with the simple idea of defining the forward pass and then letting the math take care of the backward pass (backpropagation) for all the following steps. 

Our thoughts: While it sounds simple, AD is the backbone of every current deep learning framework. AD allows us to build rapidly, test, and put Deep Learning Architectures into use.

Yes, but: The procedure's math and intricate details are always hidden. We excavate them in today's tutorial.

Stay smart: Understanding how that function works and how it computes backpropagation gives us a more profound knowledge of deep learning architectures and the superpower to tweak architectures if and when we need them.

Stay tuned for the next part of this series, where we journey through a Python library designed to tackle the task of automatic differentiation.

Click here to read the full tutorial

Do You Have an OpenCV Project in Mind?

You can instantly access all of the code for Automatic Differentiation Part 1: Understanding the Math, along with courses on TensorFlow, PyTorch, Keras, and OpenCV by joining PyImageSearch University. 

Guaranteed Results: If you haven't accomplished your Computer Vision/Deep Learning goals, let us know within 30 days of purchase and get a full refund.

Do You Have an OpenCV Project in Mind?



Your PyImageSearch Team