On the 11th of June, 2020, the world was not ready for OpenAI to release a beta version of their latest language training model, GPT-3. Here’s a quick tour of this powerful language model.
What is GPT-3?
GPT-3, or Generative Pre-trained Transformer 3 is the latest breakthrough in language generators. It uses deep learning technologies to generate human-like texts. It’s the largest language model to exist so far, with over 175 billion parameters. In comparison, its predecessor, GPT-2 has only 1.5 billion parameters.
Here’s a graph comparing the number of parameters of other pre-trained NLP (Natural Language Processing) models.
How does GPT-3 work?
With over 175 billion parameters, it’s no surprise that GPT-3 is one of the most sophisticated text predictors to date.
But how is this possible?
Let’s go back to basics for a second. Remember when we first got introduced to transformer models back in 2017? They essentially used a deep learning technique called neural networks, to handle natural language data for translations and text summarisations. This transformer became the standard for all language generators, so GPT-3 followed suit. GPT-3 models are based on the same deep-learning transformer-based neural network architecture.
It’s the same standard, but it’s larger. And is pre-trained by an abundance of resources via datasets, such as Common Crawl, Wikipedia, WebText2, Books1, and Books2. The amount of training removes the need for human intervention and saves a lot of time. It’s like having the entire internet’s history in your generator.
Every article, every resource, every piece of information, all in on one powerful machine.
And with abundance of information, GPT-3 can generate the most statistically likely response for any given input, based on what it has learnt from the internet. Rich insights can be extracted from patterns in large data sets; And it can identify and apply linguistic patterns at a speed beyond what any human can do. Think of GPT-3 as a sort of virtual assistant.
What can GPT-3 do?
The short answer is, a lot. But here’s a narrow list of what we believe GPT-3 can do.
GPT-3 is currently in beta, but we at Engati are excited for how things will evolve in the future. In the next series of blogs, we’re going to talk about the various use cases for GPT-3 in chatbots.
The future of chatbot technology looks bright with GPT-3 at our side, so stay tuned for the next one!
GPT-3's full version has a capacity of 175 billion machine learning parameters.
- Wikipedia
Until then, explore our current chatbot offerings with Engati.
Register to stay ahead of the conversational automation curve.
