Don’t Believe the Hype: Tests aren’t designed to trick you

Public Enemy - Dont Believe The HypeOver and over I read blogs and hear comments about how evil standardized tests are and how they are designed to “trick,” “fool,” and “trap” test-takers. I say that’s hogwash and poppycock!

It’s the invention of the test prep industry so they can sell you their “miracle cures.” This isn’t to say that all test preparation companies take this line. A few companies, Bell Curves among them, pride themselves on providing test prep that speaks to the knowledge, insights, and strategies needed to conquer the test, rather than play into the notion that these tests are designed to trick test-takers. My gripe with the other, more popular position is that it seems designed to make the test out to be a big scary mysterious unknowable boogie man designed to jump out of the dark and bop the unwary, and thus force test-takers to get help from someone else to defeat the unknown and unknowable. However, if the test is just a test, a test of content, a test of information, a test of factoids presented in a very particular way, then you might be able to prepare on your own. It’s got to be easier to sell a course or tutor if “only SAT experts” have the key to this very special lock.

Don’t believe the hype!

The ACT, SAT, GMAT, GRE, LSAT, et al. are not designed to trick you. They are not designed to keep you out of school. They are not designed to make you weep in the corner because you are a failure in life.

Now don’t get me wrong, these tests are tricky, they do keep people out of school, and they do make people cry. But they are not designed to do that. There is no great corporate conspiracy intended to make you and your ethnic, economic, or gender group perform poorly. These tests are designed to measure particular skills in particular ways. If you have never been trained to approach testing in the way these tests require, then these tests will present a problem. If you do not have these skills or are not prepared for how the tests measure these skills, then you will perform poorly on these tests. But that holds true for most things in life which test information that might not be intuitive, that needs to be acquired, processed, and applied. If you’ve never taken driver’s ed, then it’s likely you will preform poorly on your written or road test (Don’t believe me? I dare you to click here and take the CA DMV written test).

In addition to the challenges of access to preparation and training, these tests represent an ideology about measuring information. This is an ideology I find contrary to most of our social interactions and that presents additional challenges. Let’s consider what a standardized test is:

Standardized tests are tests designed in such a way that the questions, conditions for administering, scoring procedures, and interpretations are consistent.

Thus, in order to create an effective standardized test, it must be consistent for all test-takers. The tests we’re discussing here are administered internationally. They must be created without our regional linguistic slang/usage or social connotation. Additionally, these tests are multiple choice, which means the answers play a fairly big role in test-taker performance.

The way questions are asked must be understandable to all test-takers equally.

The only way to have reasonable hope that all test-takers will interpret written material in the same way is to use the English language carefully, literally, and precisely. This has unfortunate consequences for many since socially we do not use the language in that way. Look at the two math questions below. While the student might have been trying to be clever, I would give her/him credit for the questions since she/he answered the question that was asked of her/him. It was the the fault of the questioner for being vague.

Standardized test writers work hard to ensure that what happened to the teachers above doesn’t happen on their tests. However, many test-takers have learned to overlook and ignore the lack of specificity present in the two questions above, and instead of doing what was done above, they answer the question they believe the writer was asking, instead of the question actually asked. Just think, if you’ve been used to using “many” to mean “more than half,” then you will perform poorly on these tests which use “many” according to its dictionary meaning.

Many: Being one of a large indefinite number.

What’s interesting to me in the definition of “many” is the use of “large,” which is in itself an indefinite term. A large amount of money is very different than a large number of children. Let’s look at the second part of making a good multiple choice test.

The incorrect answers must be as carefully and intentionally created as the correct answer.

In the test prep industry and classrooms all over the country, much is made of the “trap” answers that are included in all multiple choice tests and how they are malevolently designed to bamboozle the innocent. Again I quote Chuck D: Don’t believe the hype! These tests are written to evaluate certain skills, information, and the ability to reason in certain ways and the only way to do that in a multiple choice setting is to provide not only the correct answer for a question, but also reasonable, selectable incorrect answers. Many people like to call these answers “traps” or “distractors,” but those are misnomers, as they can only trap or distract you if you either don”t understand or don’t pay attention to exactly what they say and mean. Those “trap” answers are also necessary to create an effective test. Consider the following questions:

Which of the words given best fits into the blank in the sentence:

Many ——- of standardized testing harp on the misuse of testing data, misalignment of questions to curricula, and inability to measure social skills in their assessment of standardized exams.

(A) donators

(B) parents

(C) farmers

(D) critics

(E) dancers

Anyone reading this question should be able to get it right whether they know the meanings of “misuse,” “misalignment,” and “inability,” and this question would provide no insight into the skill level, vocabulary, or reading ability of the test-taker. In order to effectively measure those skills we can make simple changes that provide the test-taker more than one selectable answer.

Many ——- of standardized testing harp on the misuse of testing data, misalignment of questions to curricula, and inability to measure social skills in their assessment of standardized exams.

(A) proponents

(B) composers

(C) malcontents

(D) critics

(E) statisticians

This version of the question plays off of the reasonable misinterpretations of the question to create answer choices, thus a test-taker would be much less likely to be able to luck into a correct answer. It also introduces more difficulty vocabulary in order to tests vocabulary skills.

So what does all of this mean to you?

If you are a parent, I suggest you help your child out by periodically holding yourself and your child to higher standards of specificity. Teach your child to “code-switch,” make them understand that certain situations require different usage of the language. Some days we’ll be very specific and sometimes we won’t. On most days “can I see that cup?” probably means that the cup gets handed over, but other days (maybe there is “specific day” in your house like there is in mine) that same question gets the response “you’re not blind.”

If you are a student preparing for a test, you should start to pay closer attention to your use of language and how it aligns or misaligns with the way the test you are preparing for uses language. Many students who struggle do so because they are too interpretive and subjective. Train yourself that critique and critics do not necessarily carry negative connotation.

If you are an educator, tell your students what will help them. Some students are motivated by the thought of the test trying to “get” them and the fact that they have to watch out for the “traps” of the evil testing companies. If this is so, then play up that angle and use that, and if that’s not the case, then don’t. For some it’s disheartening to be constantly tricked. For some the thoughts of watching out for “tricks” and “trap” answers simply makes the test-taker doubt every answer they arrive at and thus hurts their score.

For those in the test prep industry, stop saying there are boogeymen in the closet!

Good luck and good learning!

  • (646) 414-1586
COPYRIGHT ©2002 - 2020 BELL CURVES LLC. ALL RIGHTS RESERVED. All tests names are registered trademarks of the respective testing companies, which do not endorse and are not affiliated with Bell Curves.
BELL CURVES - 25 West 36th Street, 8th Floor - New York, NY 10018 Bell Curves is an educational services and test preparation company. It delivers high-quality consulting services, test preparation programs, and self-study resources to students throughout the country.
Equal Opportunity Employer - Privacy Policy - Refund Policy