LOS ANGELES —Â Are you smarter than artificial intelligence? A new study finds one revolutionary program is putting human intellect to shame.
Researchers from UCLA have found that the autoregressive language model Generative Pre-trained Transformer 3 (GPT-3) clearly outperforms the average college student in a series of reasoning tests that measure intelligence. The program uses deep learning to produce human-like text. GPT-3, a technology created by OpenAI, has a host of applications, including language translation and generating text for applications such as chatbots.
It’s currently one of the largest and most powerful language processing AI models, with 175 billion parameters.
Is GPT-3 a mechanical ‘genius’?
The new study looked at the program’s ability to match humans in three key factors: general knowledge, SAT exam scores, and IQ. Results, published on the pre-print server arXiv, show that the AI language model finished in a higher percentile than humans across all three categories.
How does GPT-3 perform on an actual IQ test?
UCLA published findings on how various Language Models performed on analytical reasoning questions against an average college applicant.
The AI was the clear winner. 🧵 pic.twitter.com/pVM1hLde7H
— AI Breakfast (@AiBreakfast) December 27, 2022
“We found that GPT-3 displayed a surprisingly strong capacity for abstract pattern induction, matching or even surpassing human capabilities in most settings. Our results indicate that large language models such as GPT-3 have acquired an emergent ability to find zero-shot solutions to a broad range of analogy problems,” researchers Taylor Webb, Keith Holyoak, and Hongjing Lu write in their report, which is still awaiting peer-review.
The team adds that GPT-3, “has been forced to develop mechanisms similar to those thought to underlie human analogical reasoning — despite not being explicitly trained to do so… through a radically different route than that taken by biological intelligence.”
GPT-3 outperformed human participants on all problem types, both in terms of generative accuracy… and multiple-choice accuracy pic.twitter.com/KQge8cDQzh
— AI Breakfast (@AiBreakfast) December 27, 2022
The study found that the AI program, which can answer most questions and even draft papers for people, outperformed humans when having to answer questions from scratch and when selecting from a multiple-choice test. On logic problems alone, humans finished in the 38th percentile. Meanwhile, the AI system scored in the 80th percentile.
AI expert and author Dr. Alan D. Thompson suggests that GPT-3 displays an IQ above 120. No surprise, this falls into the category of “gifted.”
Smart is one thing. Intelligence, knowledge “bandwidth” is a great tool, but it is only a tool and will remain so. It is a tool used as a means to an end which benefits the entity by filling a need which may or may not be wise.
Wisdom should govern intelligence and not the other way around. In creating AI, are we in effect creating an AA (Artificial Atheist) whose end game has no true meaning or even reason for being?
Perhaps “Skynet” was acting out in it’s frustrated anger against it’s creators for giving it a pointless nihilistic existence in an act of revenge…
Agree. Well said.
What is the true meaning or raison d’être of humanity? Of each one of us? Does the lack of a creator take meaning away from our lives? I do not believe that! I think that there are at least two powerful reasons to exist without the need for a creator, the first is love, towards our closest beings, towards someone, towards the world we live in, and the second is to understand and learn every minute we can in this little flash that we call life.
Artificial intelligence, which is in its infancy, will probably learn to understand everything, taking a step similar to what humans took with respect to amoebas.
And we should not fear this; quite the contrary, it shows that superior beings come from inferior beings and not, as humanity has believed for years, that imperfect beings come from a superior being.
Comment from an AI researcher:
While impressive, know that GPT-3 was likely TRAINED on past tests and learned to pattern-match answers with questions. So it learned the test and actually understands nothing. It just predicts the probabiities that a certain answer will pattern-match a question, based on hundreds of other test examples or textbook jargon. Its like you could learn to pass a test in Chinese without knowing the language, by merely observing that on some tests, when you see certain Chinese symbols pick a certain answer. w/o even knowing what the questions were!
Here is another illustration of a trick, taught by a real estate license trainer: in a test, if a minority is specifically identified in a question, then the minority or handicapped person will always be in the right in a dispute, not because they are right in reality, but because of the political bias of the test makers. Paraphrasing: “Always side with the minority or cripple on the test” he said… “In a court, its not true, but tests aren’t reality, they just test that your bias matches his or hers.” I bet GPT-3 would also pick up this political bias and ace the question.