Dr Peper
Have you ever wondered how far AI will go? Will it be able to write novels one day or replace the need for computer programmers? With a breakthrough in AI called GPT-3, these scenarios seem almost in the realm of possibility.
GPT-3 stands for Generative Pre-trained Transformer 3 (the third version). Some have called it the most important and useful advance in AI in years. The abilities of GPT-3 have both shocked and excited many within the AI community. As one developer said: “Playing with GPT-3 feels like seeing the future.”
But, how was GPT-3 developed? Find out in this episode of Short and Sweet AI.
You can listen to this episode below or keep reading.
Another Mind-Blowing Tool from OpenAI
How does GPT-3 work? It was developed by AI research lab, OpenAI, and became the largest artificial neural network ever created.
GPT-3 has learned to generate natural language by analyzing thousands of digital books, the whole of Wikipedia, a trillion words found on social media, countless blogs, and news articles. Essentially, any kind of text online has been analyzed. That’s trillions of words!
It’s a Language Predictor
Based on all the content it analyzed, GPT-3 can answer questions, write essays, summarize long text, and even translate languages. Anything that has a language structure can be replicated because GPT-3 is a language predictor.
Supplying it with just one piece of language allows the algorithms to transform and predict what should follow it.
Machine learning neural networks work by studying words and their meanings. They then compare them with other words used in texts. The machine analyzes these words to understand language and then uses this to generate sentences. It essentially takes words and sentences apart and rebuilds them.
Supervised vs. Unsupervised Machine Learning
GPT-3 is a form of machine learning called unsupervised learning. This doesn’t mean that the machine has a mind of its own.
Unsupervised learning means that the training data is not labeled as a right or wrong response. It’s free from the limits imposed by using labeled data. This means that it can detect all kinds of unknown patterns to discover new information.
In comparison, with supervised machine learning, the machine cannot learn on its own. It’s supervised during training by using data labeled with the correct answer. The problem with this method is that it’s not very flexible. It means it can’t capture more complex relationships or unknown patterns like GPT-3 can.
Judge for Yourself
How far can GPT-3 go? When OpenAI first described GPT-3 in a research paper in May 2020, it allowed selected people and developers to try it out. They reported all of their experiences of what GPT-3 can do.
There’s even an article about GPT-3, written entirely by GPT-3 itself!
Another researcher used GPT-3 to generate a Harry Potter parody in the style of Ernest Hemingway. Take a look at it below:
“It was a cold day on Privet Drive. A child cried. Harry felt nothing. He was dryer than dust. He had been silent too long. He had not felt love. He had scarcely felt hate. Yet the Dementor’s Kiss killed nothing.”
It sounds pretty good for a machine!
There’s also a Twitter account called @gptwisdom, which generates quotes using GPT-3. Here are a few examples:
“Dull as a twice-told tale.”
“The point at which a theory ceases to be a theory is called its limit.”
“The truthfulness of your simplicity can only grow as you improve your character.”
All of this is written by a machine, but most people probably couldn’t tell the difference.
Things to Know About This Technology
In essence, GPT-3 is a universal language model. It has learned to identify more than 175 billion different distinguishing features of languages.
These features are mathematical representations of patterns. Those patterns represent a map of human language. With this map, GPT-3 has managed to perform all sorts of tasks. Many of these, it was not even built to do.
Unintended Abilities
One of the unexpected abilities of GPT-3 is writing computer code. It makes sense, of course. Computer code is a type of language, after all. However, this behavior was entirely new and even surprised the designers of GPT-3 themselves.
GPT-3 was not built to generate computer code. All it was designed to do was predict the next word in a sequence of words. With this simple idea, people have discovered it can perform many tasks it wasn’t originally trained to do.
They found it could build an app just by giving descriptions of what they wanted the app to do. It can generate charts and graphs from plain English, identify paintings from written descriptions or create quizzes to help people practice any topic. It seems like there’s really no end to what it can do.
The Best but Flawed
There’s no doubt that GPT-3’s abilities are impressive and the best thing we have seen in AI for a long time. However, it’s far from flawless.
The dark side of GPT-3 is that it can spew offensive and biased language at times. It can also struggle with questions that involve reasoning by analogy.
It isn’t guided by coherent understandings of reality because there is no internal model of the world. As a result, it can sometimes produce nonsense because its primary function is to string words together. Beyond predicting words and computer code, GPT-3 struggles.
A Machine Like Us
Despite its flaws, the consensus is that GPT-3 is shockingly good. It can create blog posts, computer code, and Tweets so convincingly that many people can’t tell that a machine wrote them.
Some even suggest that GPT-3’s abilities are almost human. The risk of seeing humanity in the GPT-3 system is that it ignores the apparent limitations. There is still a long way to go for GPT-3 beyond replicating and generating text.
Sam Altman, one of the founders of OpenAI, urges caution about the hype surrounding GPT-3. He says, “AI is going to change the world, but GPT-3 is just a very early glimpse. We still have a lot to figure out.”
Perhaps it’s a little too early to suggest that GPT-3 is anything human-like. But it appears to be far more capable than anyone imagined.