Are you smarter than AI? Computer language model the clear winner over people in IQ tests

LOS ANGELES — Are you smarter than artificial intelligence? A new study finds one revolutionary program is putting human intellect to shame.

Researchers from UCLA have found that the autoregressive language model Generative Pre-trained Transformer 3 (GPT-3) clearly outperforms the average college student in a series of reasoning tests that measure intelligence. The program uses deep learning to produce human-like text. GPT-3, a technology created by OpenAI, has a host of applications, including language translation and generating text for applications such as chatbots.

It’s currently one of the largest and most powerful language processing AI models, with 175 billion parameters.

Is GPT-3 a mechanical ‘genius’?

The new study looked at the program’s ability to match humans in three key factors: general knowledge, SAT exam scores, and IQ. Results, published on the pre-print server arXiv, show that the AI language model finished in a higher percentile than humans across all three categories.

“We found that GPT-3 displayed a surprisingly strong capacity for abstract pattern induction, matching or even surpassing human capabilities in most settings. Our results indicate that large language models such as GPT-3 have acquired an emergent ability to find zero-shot solutions to a broad range of analogy problems,” researchers Taylor Webb, Keith Holyoak, and Hongjing Lu write in their report, which is still awaiting peer-review.

The team adds that GPT-3, “has been forced to develop mechanisms similar to those thought to underlie human analogical reasoning — despite not being explicitly trained to do so… through a radically different route than that taken by biological intelligence.”

The study found that the AI program, which can answer most questions and even draft papers for people, outperformed humans when having to answer questions from scratch and when selecting from a multiple-choice test. On logic problems alone, humans finished in the 38th percentile. Meanwhile, the AI system scored in the 80th percentile.

AI expert and author Dr. Alan D. Thompson suggests that GPT-3 displays an IQ above 120. No surprise, this falls into the category of “gifted.”

Comments

  1. Comment from an AI researcher:

    While impressive, know that GPT-3 was likely TRAINED on past tests and learned to pattern-match answers with questions. So it learned the test and actually understands nothing. It just predicts the probabiities that a certain answer will pattern-match a question, based on hundreds of other test examples or textbook jargon. Its like you could learn to pass a test in Chinese without knowing the language, by merely observing that on some tests, when you see certain Chinese symbols pick a certain answer. w/o even knowing what the questions were!

    Here is another illustration of a trick, taught by a real estate license trainer: in a test, if a minority is specifically identified in a question, then the minority or handicapped person will always be in the right in a dispute, not because they are right in reality, but because of the political bias of the test makers. Paraphrasing: “Always side with the minority or cripple on the test” he said… “In a court, its not true, but tests aren’t reality, they just test that your bias matches his or hers.” I bet GPT-3 would also pick up this political bias and ace the question.

  2. Smart is one thing. Intelligence, knowledge “bandwidth” is a great tool, but it is only a tool and will remain so. It is a tool used as a means to an end which benefits the entity by filling a need which may or may not be wise.

    Wisdom should govern intelligence and not the other way around. In creating AI, are we in effect creating an AA (Artificial Atheist) whose end game has no true meaning or even reason for being?

    Perhaps “Skynet” was acting out in it’s frustrated anger against it’s creators for giving it a pointless nihilistic existence in an act of revenge…

Leave a Reply

Your email address will not be published. Required fields are marked *