GPT

What are they, why are they important, and how do they impact us?

Zander Huff
March 07, 2024

Happy Little Friday (Thursday) to our favorite normies out there! All this talk about OpenAi and ChatGPT had me wondering - what actually is a GPT?

Turn Text to Video in Minutes

Paid Partnership with Synthesia

You’re hearing all the buzz about needing to make social media content but not feeling comfortable in front of the camera? Do you need to make a quick video for work but are looking for a different option?

Among other tools, Synthesia can take an image you give it with a script and turn it into a very nice video to get your message across.

Don’t miss out on what they’re building!

The What
GPT

GPT stands for Generative Pre-Trained Transformer, a type of large language model designed to generate human-like text from the input it receives.

If you’re interested in how LLMs or GPTs work—how they turn the inputs we give the prompt into digestible outputs for us mere humans to understand—feel free to read our previous newsletter on tokenization. Tokenization is that process.

The Why
Why are GPTs Better?

So, we know what a Large Language Model is.

And we know... kinda... what a GPT is.

We know that a GPT is in the large language model family subset. One thing to note: Not every large language model is a GPT, but every GPT is a large language model.

What makes a generative pre-trained transformer different from a large language model?

Now we’re cooking.

The difference lies in the “Transformer” believe it or not. How would we have ever guessed…

This groundbreaking discovery was first made in 2017 in the paper “Attention is All You Need” by Vaswani et al.

The biggest thing to note about the transformer model is that instead of the model having to read a sentence input in sequence (left to right) (which takes a lot of time and a lot of data), we can now use the self-attention mechanism (basically having a model read a sentence all at once) to recognize the weight of each word within the sentence, all at one time, regardless of the distance between words in a sentence.

In common terms, it can read a sentence at once (an AI’s computing power will allow it to do this; a human brain cannot), digest it, and respond accordingly.

This is vastly important for the speed and efficiency of training models. It is a massive reason why OpenAI is moving through its updates quickly, from 3 to 3.5, 3.5 to 4, and apparently 4 to 5 pretty soon.

The How
How does this impact me?

This allows for a couple of things that directly impact you and me.

1) More parameters in a dataset.

The more data parameters that the model can learn from, the better the model will be for you and me in its responses. Do you like AI responding for you in your emails? Do you like ai knowing the best sweet potato casserole recipe? Do you like ai knowing the last 4 digits of the number pi? pretty sweet.

More data also leads to a chance of less bias. This will drastically benefit professional roles in politics, financial platforms, and corporations all over America. The less biased, the better; it will generally reach the country and world.

2) Quicker speed on Updates

The reason why we have ChatGPT 4 on the verge of 5 already? The transformer architecture. Without this, ChatGPT might have just released version 3 last week…

You’ll continue to see faster and faster updates at an exponential rate, not linear (assuming the benchmark for an update stays the same… which is probably won’t because it would become numbing in a few years when you have a new update every 2 weeks).

Wrap it up
In Conclusion

GPTs aren’t going anywhere for now. This is a breakthrough in technology that happened a few years ago that streamlined the movement of AI. I'm excited to see the future of what we will get to build.

Enjoy the weekend - See you on Sunday!

Zander