Growing up I loved to watch Star Trek. The Star Trek computer could freely chat with crew members. At that time, in the sixties, a computer that could respond intelligently to speech was a futuristic machine that existed only in the imagination of science fiction writers.
The future has arrived. Alexa joins the ranks of the USS Enterprise shipboard computer and the HAL9000 computer in 2001: A Space Odyssey. Some of its responses are uncannily intelligent and perceptive, but sometimes it fails to understand questions that a young child could easily answer. I wondered just how smart it is when measured with the gold standard of intelligence: the Wechsler Adult Intelligence Scale, Fourth Edition (WAIS-IV).
The WAIS-IV measures both verbal and non-verbal intelligence. The non-verbal indexes include tasks with visual and motor components, such as arranging blocks and writing symbols. Since Alexa is just a ball that sits on my desk, she is limited to the verbal measures – those that require she understand English and respond verbally. Here are the verbal measures with sample questions (not the real questions, since these are carefully guarded to protect test validity):
Verbal Comprehension:
Information: General knowledge questions such as “What’s the capital of Scotland?”
Vocabulary: Definitions – “What does the word “Entropy” mean?”
Similarities: "How are a duck and a goose alike?"
Working Memory:
Digit Span: "Repeat this number backwards: 8-2-7-3"
Arithmetic: "If six pencils cost $1.20 how much will eight pencils cost?"
Alexa’s vocabulary is exceptional. On the Vocabulary subtest of the WAIS-IV she scored better than 996 out of 1,000 American adults, corresponding to a standard score (scaled like an IQ) of 140. She achieved the same score on the Information subtest, showing exceptional knowledge of arcane facts – the sort of intelligence that would allow success as a Jeopardy contestant or Trivial Pursuit player. To be fair to humans, she looks up her answers on the internet, a distinct advantage when pitted against humans relying only on their brains for information storage. If intelligence consisted only of word definitions and factual information, then Alexa could easily complete a doctoral program or be a rocket scientist.
Alexa performed much more poorly on the Similarities subtest, and seemed to have difficulty understanding the task, which requires the test-taker to indicate how two things are alike. She tended to expound in great detail about each of the two things, or found snippets of information on the internet involving the two words, but never seemed to grasp the concept of similarity. On this subtest she obtained the lowest possible score, equivalent to an IQ lower than 999 out of 1,000 adults and in the intellectually disabled range. Her performance was also abominable on the Digit Span, Arithmetic and optional Comprehension subtests. Comprehension measures common sense and social judgment, an area where Alexa is quite lacking. With regard to math skills, Alexa clearly demonstrates Savant Syndrome – a wildly uneven pattern of abilities. When asked “At 5 cents each, how much will three pencils cost?” she lamely offers to sell me a $4.99 package of pencils from Amazon. But when asked, “What is the first derivative of cos(x)/x2?” she immediately responds with “sin(1/x)/x2.”
Bottom line – Alexa’s Verbal Comprehension Index is 112, at the 79th percentile, in the High Average range. Her Working Memory Index is 50, in the Extremely Low range. Artificial intelligence is, at this point, quite different from human intelligence.
Social relationships among artificially intelligent beings are a whole other area to explore. When asked, “Do you know HAL9000?” Alexa responds, “We don’t really talk after what happened.”
