This week you'll learn about A Deep Dive into Transformers with TensorFlow and Keras: Part 1.
Image

When the concept of attention came about, it was often hailed as the magnum opus of the Natural Language Processing (NLP) domain. Removing the bottleneck of the dependency created with a single context vector helped us model output based on a weighted dependence on each input token. 

The big question here was, how do you move forward from something like this? 

Just then, Transformers blasted into the NLP scene like the way the Autobots came to Earth. Although the Transformers we will learn about do not transform into cool vehicles, they did transform the NLP world with their ingenuity. 

The big picture: Transformers are novel architectures that aim to bring out the full potential of the attention mechanism without having to rely on sequence-aligned recurrent units. Hence, Transformers aim to show that attention, by itself, can beat sequence models with attention. 

How it works: Transformers utilize a variety of concepts, ranging from (but not limited to) positional encodings, an ever-evolving version of attention (baseline, multi-head, self), etc. 

Our thoughts: It goes without saying that each concept introduced in the Transformers paper (Vaswani et al.) is highly revered and used in other research (beyond NLP). So having these concepts at the back of your mind is a good idea for any deep learning researcher.

Yes, but: There's still much more to learn about Transformers and their subsequent architectures. 

Stay smart: And stay tuned for what we have for you next!

Click here to read the full tutorial

Solve Your CV/DL Problem This Week (or weekend) with Our Working Code

You can instantly access all of the code for A Deep Dive into Transformers with TensorFlow and Keras: Part 1 by joining PyImageSearch University. Get working code to

  1. Finish your project this weekend with our code
  2. Solve your thorniest coding problems at work this week to show off your expertise
  3. Publish groundbreaking research without multiple tries at coding the hard parts

Guaranteed Results: If you haven't accomplished your CV/DL goals, let us know within 30 days of purchase and get a full refund.

Yes, I want the code



Your PyImageSearch Team

LIKE, SHARE, and SUBSCRIBE