The Problem with IQ Tests
Intelligence is distributed evenly throughout the global population. Opportunity is distributed unevenly, mediated by wealth. This is a huge waste of humanity's potential.
Recently I took an IQ test. The result stated I have an IQ of 177. In my opinion the test result was flawed and inaccurate. At the same time, many people and organisations place great store in the result of IQ tests. This is a mistake.
Over the years I have never seen an IQ test like the Ravens Progressive Matrices. Many decades ago, when I joined a new school the New Boys were shuffled into a classroom and told to sit down for one hour to do an IQ test. At the time, I didn't have the slightest interest in doing either the test, or of sitting down for an hour. Besides, I had other things on my mind. I made a perfunctory stab at some questions, watched my fellow pupils arduously work, and I wondered why the invigilator, a teacher I would grow to learn and respect, wasn't paying attention to the class. I wanted to understand why he wasn't watching us. Perhaps he didn't want to bias the result by introducing an external influence. The test was soon over and the results were kept "secret." It was that kind of school.
In those days, IQ tests consisted of questions written in English. Invariably they contained ambiguities and in some cases, phrases that I found upsetting. I thought they couldn't really be IQ tests since the design was so poor, and I thought they were stupid. Such is youth.
Jump forward to yesterday afternoon and settling down after lunch without specific aim, or guided by my furious subconscious, it's hard to say which, I found myself on a website that described the Ravens test. It made me curious. It was designed to de-emphasize culture and not to depend on language. It was a visual type of logic test. That sounded interesting despite my misgivings that if I was to do an IQ test, verbal reasoning is probably an area I would do better in. The Raven test might not favor me but it looked fun. And I was pissed someone I knew had equated poverty with stupidity. The idea that people who are poor must be stupid makes me particularly furious since I spend most of my time in developing countries. If people are poor they lack opportunity. As far as I know intelligence is distributed normally, in the statistical sense.
The test was easy. Too easy. I guessed the first two pages were designed for children since they consisted of very simple pattern matching problems. If you can complete a classic wood puzzle, these questions were about the same level. Afterwards came questions with slightly different mechanisms. They were additive or subtractive. Or there was some other manipulation - one had to find the relationship - between the symbols that formed the question.
The preamble said, "Take as long as you need. There is no time limit." I wasn't in a hurry, so I got up to get water and to go to the bathroom. I took my time. There was no rush.
In retrospect I suspect all IQ tests need to have a time constraint which in itself raises questions about age both chronological and biological and how the latter should be measured, and leads into yet more questions about the myriad of confounding factors that can influence test-taking of any kind at an individual rather than generalized level.
By the last page I was waiting for more difficult questions. But still they didn't come. It was only in the penultimate question that I could perceive some level of difficultly. I couldn't intuit the answer as I could with some of the earlier questions, and I couldn't work it out either. I couldn't reach a solution I was happy with. I got up, took a drink of water, went back to my desk, and looked at the question again. Nah. Not sure. I still wasn't "getting" it. I wanted to finish so I selected what I thought was the most likely answer. And scrolled down.
I had reached the end of the test. Which is where things went a bit wrong. My guess is the Javascript blocker NoScript in my Firefox browser, or one of the other security Add-Ons, interfered with the test. The last six questions were repeated as I scrolled down the page, and then there was a button marked, "Submit" or its equivalent. I clicked through the questions filling in my answer again, and pressed the button. The page returned a dialog box with the message that some of the questions hadn't been answered. I scrolled to the top of the page to find my previous answers had disappeared. I selected those answers again and scrolled back down the page. I figured I would press the Submit button to see if any other questions had magically disappeared.
The website returned my score. It was 122. There were a few platitudes, perhaps I could make a managerial position. It didn't seem right. I suspected that the last six questions hadn't been counted. So pressed "Start Again" and re-entered what I could remember of my answers to the test.
On the second attempt the same thing happened, only this time the last page repeated all the questions E1 to E12 twice. I filled in the second set, pressed Submit and the system responded with a message to complete the questions. I went back to fill out the first block of E1 to E12, pressed Submit yet again, and got a score of 128. It still didn't seem right.
There was something wrong with the way my modified version of Firefox was interacting with the IQ test Javascript or website server. So I fired up a Chromium web browser and went back to the IQ test to complete it for a third time. As I went through the questions, I had to answer them again since by this time I couldn't remember my first answers. I wanted to be quick since the test was taking up too much time and I needed to get some work done. I should note that when the system returns a score, it doesn't tell you which answers are right or wrong. So I still didn't know the answers. Nonetheless, I think there was advantage through happenstance. I reconsidered a couple of questions as I re-entered what I could remember of my previous answers, and re-thought the answers for those that I couldn't remember. Or, now I was concentrating. I was in a hurry to complete all sixty questions.
One question showed a series of icons that formed a pattern. The logic of the question was to see that certain icons had changed shape by a consistent amount and the answer therefore was to select the final icon according to a similar reduction (in this case of 50%). What was interesting from a question design point of view is that earlier I had responded to this question aesthetically. In other words, rather than address the question as a logic problem, I had replied with what looked prettiest.
It occurred to me that though I admire the idea of an IQ test that removes cultural bias, and believe me when I say I think this is of crucial importance, there are always unexpected consequences. In this case, there were three questions where my first instinct was to select what looked best in terms of the patterns of icons in the question. I preferred the most aesthetically pleasing solution. I had to consciously remind myself that this was a logic test therefore a logical answer was required not a visual one.
At heart, there is a conflict in the question design. To a naïve examinee like a child I think an aesthetic answer is a perfectly good answer. After all, if no one has explained logic, why not choose the best "looking" answer for some of the questions? People being people, there will always be variation in outlook. Would it be fair to expect an artist (or a photographer!!!) to respond to a visual IQ test with visual replies? I think so.
It was with some relief that I got to the last page of twelve questions and then the final two questions. It was here that my answer to the penultimate question still didn't satisfy me. It was more of a guess than a reasoned answer. Later, when I reviewed the questions after the test, the answer to that question came intuitively. One of those, I can't believe I didn't get it sort of moments. The subconscious is a powerful thing. I pressed, "Submit." The following page asked for my age, and after entering it a result was returned; my IQ is 177.63157894737.
Eleven decimal places. Ridiculous!
There was some text attached to the result which I have issue with. It commented on creative and logical abilities and stated they, "can help you succeed in any industry." It went on to state in non-English English that this level of IQ was similar to well known people - how would anyone really know their IQ scores - and it stated, "you are able to make a high contribution to the development of knowledge and science." This was very disappointing.
One of the critical problems with IQ tests is what is called determinism. In short, if a child is lead to believe they are clever, they tend to perform better. They may be more motivated with their schoolwork, and soon the compounding effect of education takes hold. Similarly, and tragically, if a child is lead to believe they are not clever they may never develop their potential.
In European and North American societies that went through the 18th century Age of Enlightenment we hold an ideal of our ability to choose freely that resonates regardless of modern scientific conceptualizations of free-will. In the tension between self-direction and predestination the cult of individualism favors the illusion of an all-powerful self no matter how much evidence illustrates an energy equilibrium within a nature/nurture construct.
Contemporary discussion of IQ tends to fall into line with a simplistic way of thinking; that individual effort - a self-justifying representation of good - will overcome all obstacles. The adroit reader will recognize the popular corollary, the false notion that a failure to self-determine must therefore be a representation of evil. In other words, the erroneous belief is that if someone is not "successful" they must be lazy, feckless, or in some defining manner bad. As a result society sees a decline in humanity and compassion. It has long been recognized that in an oligarchy the notion flourishes that poor people must be bad, quad erat demonstrandum, or at least deserve their fate. "It is their own fault," says the oligarch and all those who support the oligarchical society. Which is where we are now.
In Book VIII of Republic the Ancient Greek philosopher Plato described both the education of men and women equally for the common good, and in contrast the inevitable corruption of an oligarchy. He defined an oligarchy where wealth, fame, and power, were sought above that of virtue and where wealth is accumulated through pernicious ways. Purely rational, selfless political theory is given up for self-interest and the education in courage, wisdom and moderation required for a just city state is forgone. The law becomes skewed to favor the interests of the rich, meritocracy is overruled, and the distribution of political power falls to wealthy yet incompetent leaders. In the oligarchic state economic inequality is accentuated as a result of the greed of the leading class. According to Plato the political result is that the state becomes unstable. In such circumstances meritocracy becomes irrelevant and there is a symptomatic decline in social mobility.
Outside school how many cases does one see where the determined man or woman succeeds where the more talented person fails? So many. At an individual level is it critical to know that effort over time will provide the best outcome. Not necessarily "success" as described by millions of self-help books set to enrich their authors, but the best that can be obtained from the resources, circumstances, and serendipity at hand. Sometimes a good day is staying alive.
Work - getting things done - requires many different skills. In commerce, manufacturing prizes the ability to do the same thing repeatedly to maximize the compounding effect of investment. Consider the Honda Super Cub, a 100 million units sold design little changed since 1958. Radical re-invention was not required. Or think of investment itself where a simple, consistent approach is a prerequisite to compound returns over time. A field where one of the most celebrated practitioners, Warren Buffett, argues that a high IQ is a disadvantage. As he described it, people with a high IQ start getting creative.
In the field of commercial innovation generally modest iteration is required. Genuinely new ideas are not welcome since they effortlessly represent an existential risk to the status quo. Hence large organisations filter out creative, artistic innovators. When most industries have systems and processes optimized for average levels of intelligence, high IQ is largely irrelevant to job performance.
In other areas, effort is needed beyond what one might rationally perceive to be reasonable. Olympic athletes succeed through relentless and single-minded training and practice. Artists and classical musicians and poets and novelists and horse riding show jumpers put in thousands of hours to master a technical discipline. In military Special Forces selection is designed to find the most motivated men. Those who won't give up in hard physical conditions beyond their experience, not necessarily the most obedient men. If success is broadly measured, there are many avenues none of which are reliant on a synthetic notion of intelligence.
Motivation is all important. Hence I can't shake off the idea that the risk of discouraging a pupil isn't worth the value of an IQ test, especially when IQ tests themselves are flawed at an individual level along so many dimensions. I should note that I am not calling into question either their statistical validity (although I would caution any statistician to examine and re-test their assumptions), nor their use across a population which may assist in the allocation of resources. But I would caution against the creation of a self-fulfilling prophesy.
Back to the Raven test and there's another observation I'd like to make. Instinctively I answered each question without referring to the answer beforehand. It seems to me that to look at the multiple choice selection is to bias one's thinking in an unhelpful manner. I find this technique also calms one's thinking. Whether it would have a negative influence in a timed test is open to question.
I have been aware for some time that psychologists posit that relatively high performance in one intellectual dimension, for example math or language, is more than likely representative of an individual's performance across all mental tasks. Even reaction time to a changing symbol on a screen is said to have a positive correlation with intelligence. I doubt the occasional highest possible score in a school test, or a strong showing at Trivial Pursuit qualifies. Similarly, an HPS in the Raven IQ test would seem to show up a weakness of a test designed to measure around the average, by definition an IQ of 100.
While the likes of Jordan Peterson do the world a disservice with the factual assertion that IQ is a key determinant of lifetime earnings or job outcome, they are merely reinforcing the notion that earnings is the key measure of a lifetime. Which is laughable. Two thousand five hundred years ago, Plato recognized that it is in an oligopoly that "work" is elevated over virtue as a noble goal.
Statistically derived IQ tests are generalizations. They don't work at an individual level. Imagine the effect of emotional turmoil. It is quite unlikely that a student who has just arrived at school from a home wrecked by parents fighting to destroy each other will focus on the IQ test before him. While he or she might read the questions, most of the young person's mind is trying to decode the parental clash of the night before, and fighting the dread of getting home later that day to a new, unpredictable chaos. Similarly, what of the student recently bereaved? Can he or her hold their concentration throughout a timed test? What of the girl who has accidentally been impregnated by her best friend? Intellectual pre-occupation at an individual level is neither revealed nor adjusted for by a conventional IQ test.
It is critically important to remember that IQ tests are not destiny. If you have done an IQ test and don't like the result, forget it. It's not important. Let your determination not someone else's generalised determinism be your guide. If you've read this far, there's one thing to know. You're darn persistent.
Would I want to take another IQ test with a better design? That's more interesting but I would prefer just to do the hard questions. Wading through lots of easy questions is boring to the extent that doing fifty-nine dull questions for one stimulating question isn't a very efficient return.
What was the result of that school test all those years ago? I discovered later that my score was 114. We were "streamed," that is separated into classes, "according to ability." Which didn't go well. I got very bored and dropped out of school. I was 14.
James Patrick
--
I should add - If you are in a difficult situation, there is only one option. You need to get out of there, and get as far away from it as possible. Don't stay because you think you can "solve" it. I know, lots of people will say it's your duty or your responsibility or you are a coward if you go. But you are not a coward. It takes a level of courage which makes an IQ test result trivial in comparison.
