HiLine wrote:
The ESR's that you have posted do not support that conclusion.
What you are doing is comparing two scenarios that are not equivalent.
A candidate who answers 3 questions wrong at the beginning and answers the rest of the questions correctly is going to score less than one that answers 3 questions wrong at the end and answers the rest of the questions correctly, because the latter tackles more challenging questions at the same level of accuracy and thus should be deemed a more competent candidate.
What I am hearing from most of you who disagree with me is that "test-takers who do well on the first quarter of questions are better GMAT students who earn harder questions, and thus deserve better GMAT scores". Overall, of course this is true, but it does not change my point about the unfairly "front-loaded" nature of how GMAT scores are determined.
My point is that 9 questions is not large enough of a sample size to fully determine someone's "initial" GMAT skill level, and hence there is an element of luck involved in scoring well. Answer fewer than 5 out of the first 7 counted questions correctly, and your chances for an elite score are almost zero. Throw in the fact that 2 of those first 9 questions are experimental and don't count toward your score, and this adds even more randomness into the equation. You could answer 7 out of 9 correctly, and the 2 you got wrong were experimental (100% correct), or you could get 7 out of the 9 right, but 2 of the correct answers could have been experimental (5/7 = 71%). This is of course not only true about the first quarter of the test, but the entire GMAT, where slightly more than 25% of the questions are experimental.
Yes, it is true that correct responses (normally--see below) earn harder questions, and consistently answering medium/hard questions correctly is the best path to a great GMAT score. But the student won't even get the chance to try hard questions if the GMAT algorithm makes early, "snap" judgements about his/her skill level, which it does. Anyone who is out there reading this: don't take my word for it--take a look at the ESRs and decide for yourself! The evidence is readily available if you're willing to take a look:
https://gmatclub.com/forum/information- ... 21111.htmlAny statistician knows that in order to provide an accurate measurement across a large population, you need a large sample size. A score estimate based on a small sample size of GMAT questions is more prone to fluctuations based on lucky / unlucky guesses and other random elements. But perhaps this is exactly what the GMAC wants, so that you will keep paying $250 to take the GMAT again and again.
Also, the first questions are not easy difficulty--they are medium difficulty.
What's interesting about the first 10 questions "myth" is that the GMAC does not directly dispute it...instead it just uses vague, roundabout language such as "all questions count." Yes, all questions count, but that doesn't mean that earlier questions don't affect your score more strongly than do later questions, because in fact they do, and the data from the ESRs supports this.
The excerpt below is taken directly from the
OG 2016:
"Myth -vs- FACT–
M - The first 10 questions are critical and you should invest the most time on those.
F – All questions count. It is true that the computer-adaptive testing algorithm uses the first 10 questions to obtain an initial estimate of your ability; however, that is only an initial estimate. As you continue to answer questions, the algorithm self-corrects by computing an updated estimate on the basis of all the questions you have answered, and then administers items that are closely matched to this new estimate of your ability. Your final score is based on all your responses and considers the difficulty of all the questions you answered. Taking additional time on the first 10 questions will not game the system and can hurt your ability to finish the test."Like the rest of the GMAT, this text requires a close reading. Notice that
not even the GMAC disputes that the first 10 questions have a larger impact on your final score than does any other group of 10 questions. Yes, spending too much time on the first 10 can hurt your score if you don't do so correctly (you need to save time for the remaining 27 questions, of course), but this does not change the fact that doing well on those first 10 questions is essentially mandatory for a top GMAT score. GMAT calls it an "initial estimate," but this is simply a euphemism for a snap judgment. Yes, adjustments to one's score continue to be made throughout the test, but the "Average Difficulty" charts on the ESRs support the notion that the largest adjustments in difficulty/scoring occur early in the test, and a close reading indicates that the GMAC does nothing to directly dispute this notion (although it tries its best to imply that this is untrue).
For example, the below performance of (broken down by quarters) 100% / 100% / 57% / 29% on Quant earned a 47 out of 51 (also mentioned in my original post). Not bad for getting
more than half of the final 14 counted questions wrong.
To some perfectionistic students, a score of Q47 would be a disappointment. But to many others, it would be a dream come true. This ESR shows that you can essentially turn the GMAT Quant into a 25-question test, take aggressive guesses on the final 12, and still achieve an above-average Quant score.
Should you "invest the most time" on the first 10 questions? Of course not--that's barely 25% of the questions. The GMAC is setting up its own straw man here, to avoid addressing the real issue at hand. You should not spend
most of your time on the first 10 questions, but you should spend
more time per question on those questions than you do on the remaining ones, given their outsized importance to the scoring algorithm. It is true that for elite scorers, the questions will get harder and harder, and you might suffer at the end due to timing issues, ruining your chance for a 50/51 on Quant, or something above 45 on Verbal...trust me, I've been there. But this only separates the 730 scores from the 770s, or the 750s from the 800s, and all of those scores are great. To crack 700,
leveraging the GMAT algorithm is key.
Also, you mentioned in your original post (heading #3) that the difficulty level seemed to drop, despite the student answering questions correctly, and that you weren't sure why.
HiLine wrote:
This is the conclusion that puzzled me the most:
Sometimes, when you're doing too well, e.g. answering all questions correctly, the difficulty of questions drops.
I think that this passage (also from the
OG 2016) should clarify the issue for you:
"Myth -vs- FACT-
M – Getting an easier question means I answered the last one wrong.
F – Getting an easier question does not necessarily mean you got the previous question wrong. To ensure that everyone receives the same content, the test selects a specific number of questions of each type. The test may call for your next question to be a relatively difficult problem-solving item involving arithmetic operations. But, if there are no more relatively difficult problem-solving items involving arithmetic, you might be given an easier item. Most people are not skilled at estimating item difficulty, so don’t worry when taking the test or waste valuable time trying to determine the difficulty of the questions you are answering."