Importance of the First 10 Questions - GMAT Prep What If Scenarios

A long time ago, in a couple of places (an interview with Lawrence Rudner, their chief psychometrician, and in an online article, if I recall correctly), official GMAC reps explained that the penalty for not finishing was "proportional". This was back when the Quant section had 37 questions, so if you didn't answer three questions, say, your 'score' was multiplied by 34/37. The problem though is that they never specified just what 'score' was being multiplied by that proportion -- the scaled score, the percentile, or the raw internal score (which you never see, and which is between about -3 and +3) that the algorithm uses? They do give one numerical example of the penalty in the OG, in a "Myth vs Fact" bubble (I have OG2018 in front of me, and it's on pg 13). There it says "failing to answer five Verbal questions.... could reduce your score from the 91st percentile to the 77th percentile". When they first published those numbers, a V41 corresponded to a 91st percentile score, and a V36 to a 77th percentile score (the percentiles have changed since then), and that made me think they were applying the 36/41 penalty in this case directly to the scaled score of V41 -- the numbers just worked out perfectly. But it's impossible to be sure of a conclusion drawn from just one data point, and I've seen score adjustments since then that have led me to think the penalty probably isn't applied to scaled scores.

Long story short, I'm not sure of the exact answer, and no one else will be either (outside of the GMAC offices). I also haven't seen any official comment about the penalty since the test was shortened (from 37 Quant questions to 31 and similarly for Verbal) -- it's certainly possible they modified the calculation of the penalty when they abbreviated the test. I'd guess that your estimate, of three percentile points per unfinished question, is going to be quite a good estimate for test takers who are in the average to above-average range. Because the penalty is proportional, it is less severe for low-level test takers (if you're reducing a score by 10%, the reduction is smaller for a Q20 test taker than for a Q50 one). So your estimate might not work well for test takers at extremes of the scoring range, or in extreme situations where, say, a dozen questions are left unanswered -- but I don't know for sure, it's also possible that it will be perfect in those cases! No one (as far as I'm aware) knows the exact answer, and it's unfortunately impossible to test using GMATPrep tests or the official online tests (because there's no way to know, after, say, 25 questions, what score the test thinks you have -- some people might think they can test the penalty by answering every Quant question perfectly, to get a Q51, but the Quant scoring scale goes above Q51 internally so that doesn't work).

Was going through the GMAC blog and 1 post caught my eye; has anyone else seen or analyzed this post?
https://www.mba.com/articles-and-announcements/articles/study-for-the-gmat-exam/tactics-and-guessing

Important excerpt (though the analysis is based on the older format):

Thx for posting. I think this is a repost of a much earlier presentation or article from GMAC.

It is a little conflicting and contradicts itaelf in a few places (such as it says it is impossible to determine the impact of the last 5 questions but then says just guess) but my guess you found it surprising about the last questions?

I am just not sure about how to respond to your post is it a question or point that supports scenarios or contradicts them?




Posted from my mobile device

I've read that report a few times before (it's quite old now, but its findings should still apply to the current test). I don't think it contradicts itself at all. It's impossible to know what the impact of guessing at the last few questions will be on a given test, for two reasons:

#1: you can't know which guesses will be right
#2: you can't know what the difficulty level of the last few questions will be

Point #2 is the reason they needed to study the question empirically, rather than by simply doing a pure math calculation (e.g. an expected value calculation). Even if two test takers perform identically up to say, question 26, the difficulty levels of questions 27-31 could be very different on their two tests. The test does try to adapt to a test taker's level, but it also uses other criteria to pick your questions, and sometimes you can see questions that count which are very easy or very hard for you. If a test taker is at, say, a Q49 level, and guesses randomly at their last two questions, those guesses will barely hurt at all if the questions are Q50 level (and might even help if one of them is right). But they could hurt a lot if one of the questions is a 400-level problem, since getting easy questions wrong is very harmful to your score. It's not that likely a Q49 test taker will get such an easy question at the end of a test, but it's certainly possible.

So all that report is saying is that it's impossible to say, about one specific test, what the best strategy will be. But statistically speaking, if you're a higher level test taker, it's most often better to guess and finish the test than it is to leave questions unanswered. That makes sense, since the penalty for not finishing is proportional, so it reduces high scores more than low ones. For a low level test taker, it is not true nearly as often that randomly guessing in order to finish will be helpful.

I am interested to know the outcome of scenario #1.

Can an expert advise me on what if I attempt the first 2 RCs, ensuring that I reach the first 15 questions and later guessing on the 3rd and the 4th RC (if any)? Since I am not good at RC, but good at both SC and CR, and considering the time issue, I wish to attempt all SC and CR questions. Leaving the 3rd and the 4th RC (if any) is the only way. But will employing this strategy lower my score compared to attempting all questions in order and when short of time (usually 5 questions), guess on those questions?

Suppose the following things were true:

• a test taker knew going into Verbal that he or she would need to guess at about 8 questions in order to finish
• on a fair cross-section of official questions, that test taker had a hit rate of 80% on SC and CR, and a hit rate of 30% on RC

then that test taker will almost certainly do better by guessing at two RC passages, and answering every SC and CR question properly. That would normally produce 4 additional correct answers (vs the 'worst case alternative' of answering all the RC, and guessing at eight SC and CR questions at the end). Difficulty level matters of course, but four additional right answers is pretty much always going to improve your score. I'd expect someone to gain between 1 and 3 Verbal points on average, in this exact situation, by guessing at RC instead of at random questions at the end. And it would be better to guess at the later RC questions, because they're more likely to be higher-level in this situation.

If instead someone only needs to guess at 4-5 questions, then with these hit rates, it very likely makes more sense to guess at just one RC passage, since you wouldn't need to guess at two entire passages to finish on time.

There are three considerations though:

• on a specific test, there's no way to know what strategy will be best without knowing about question difficulty. If you get RC passages that are easier than normal, guessing at them will hurt you more than normal. On some tests, this RC-guessing strategy will not work out in your favour, and there's no way to know when it will and when it won't. On average though, in this precise situation, it should work out favourably;

• I would expect that it's almost never true that a test taker has a true 'hit rate' of 80% on SC and CR, and only 30% on RC, when using a fair (i.e. covering all difficulty levels) sample of official questions. A 30% hit rate is barely better than random guessing. But RC and CR test moderately similar skills. Someone able to answer 80% of official CR questions correctly will, in most cases, be able to do much better than a random guesser at RC. So the above strategy guidance might be worthless in any practical situation. If someone told me they had an 80% hit rate at CR, and a 30% hit rate at RC, I'd want to know how they had measured that. Did they use official questions? Did they control for difficulty level? How big was their sample size? If the hit rates are closer together than 80% and 30%, the above advice could change.

• Regardless, anyone who needs to guess at 5-8 Verbal questions in order to finish is going to have a hard time getting a good Verbal score. If in addition, that person has a 30% hit rate at RC, I'd be surprised if that test taker could exceed a V32 or so no matter what strategy they used (and I would not be surprised if they got a V28 or something like that). Really rather than thinking about how to optimize timing strategy for Verbal, such a test taker would raise their score much more by improving their Verbal speed (which I know is hard to do) and improving their RC accuracy (which is probably going to be achievable with an 80% CR hit rate).

I'd add also that for higher level test takers in Verbal, it is almost never a good idea to guess through an entire RC passage, just because RC doesn't adapt by question, and any passage will have an easier question or two attached to it. Guessing at easier questions is not a good idea for higher level test takers. When a higher level test taker needs to guess in order to finish (and if the test taker is higher level, he or she presumably doesn't need to guess at 4-5 questions) it's usually best to guess at SC and CR near the end of the test, since those are the questions most likely to be difficult, and therefore are the questions that will be least harmful to get wrong.

What would you say on getting last 10 questions wrong vs missing last 10 questions. Which would be better. If getting the last 10 questions wrong is better, can i go ahead and mark all the 10 questions option B?

The best answer is "neither" -- you can't get a good score doing either of those things. But if those were your only two choices, it depends on your level. Someone who did well on the first 2/3 of the test should definitely guess at the last 10 questions in order to finish. Someone who did not do well on the first 2/3 of the test will probably get a better score by not finishing. The penalty for not finishing is higher for high-level test takers, and the best strategy to use in this situation depends on your level.

I'd add: getting all ten questions wrong at the end is probably worse than leaving them unanswered, no matter what your level, but if you guess randomly at the last ten questions, you'll rarely get them all wrong (that only happens about 11% of the time). The majority of the time you'll get at least 2 of them right.

But you'll do better guessing quickly earlier in the test at very hard questions, and saving time for later questions you can answer, than you will just by attempting everything in the first 2/3 of a test and guessing at the last ten.

Hi bb, IanStewart

These scores are from GMAT Prep Test 2.
I got 3 RCs (10 questions) in first 11 questions and got 6 of them wrong. Got stumped by a particular set. Also, got panicked as time was moving but I wasn't. Though at the end, I had 6 mins left
Got 7/13 RC, 11/14 SC, 9/9 CR correct and the overall score was just V29.

Does this verify the theory that we can't afford to falter in 1st 9-10 questions and that a high weightage is given to RCs?
Also, how rare it is to get 3 RCs in 1st 10-12 questions? And what strategy should one follow in that case?

Thanks!

Hi. I don’t think the first 10 questions will guarantee you getting a high score but as this topic illustrates, you can’t make too many mistakes in the first 10 questions because you just won’t get a high score. Again, getting the first 10 questions correct or near correct is important for a high score but it does not guarantee it. You have to maintain your performance throughout the test.

In terms of getting multiple reading comprehension questions in a row or rather passages, the sequence of questions is random so the probability of having multiple of them coming to row is whatever the probability is of not having them come in row we did have one of the dBrief‘s mentioned that they got three or four reading comprehension passages in a row on the real test. However, it seems it is fairly rare to have that experience. I frankly think it would throw me off, just because it was it up so much time at the front. I think ideally, you would have reading comprehension passages spaced throughout the test.

Posted from my mobile device

No, neither of those things is really true. You can't draw conclusions about how the algorithm works in general from a single test. What you can draw conclusions about is how difficult the questions were on that test. So on your test, it's likely the questions early on weren't too difficult. When you get several of those wrong, the algorithm will have quite a bit of evidence that you're not a medium-level test taker. So you probably got easy questions for a while afterwards, and you had to get a lot of those right to recover. If instead your test had a lot of hard questions early on (which can happen), and you got several of them wrong, the test would only think you're probably not a high-level test taker. But if you then got most of your medium-level questions right after that, the algorithm would become persuaded that you are a medium-high or high-level test taker. Question difficulty is what matters, not only hit rate, and question difficulty varies a lot from one test to the next, so what was true on this one test you took may not be true on the next.

And one 500-level (say) Verbal question is "worth" the same as any other 500-level Verbal question. It doesn't matter if it's SC, CR or RC. The question types aren't weighted in any way. There is a difference between RC and the other two types of question, though. RC adapts by passage, not by question -- if the test gives you a "500-level" RC passage, that passage might have two 400-level questions and two 600-level questions. So if you guess at RC questions, or if you get some wrong that you wouldn't normally get wrong, it's more likely you'll be getting easy questions wrong than when you guess at SC or CR (which are more likely to be at your level). And if you get easy questions wrong on an adaptive test, that hurts your score the most. Of course a random RC question is also more likely to be above your level than a random SC/CR question -- RC question difficulty is just going to be more variable than SC/CR difficulty -- so it all depends which specific questions you get right and wrong.

It's still a good idea to try to do well early in the test, as long as you don't consume hours of time doing that. If you're at an above-average level, it's more likely you'll have manageable questions early on than later on, when the test starts challenging you with hard problems. So relax early, don't rush or panic about time, and answer as well as you can. If you do get a lot of RC early, then when you're looking at the clock, account for the fact that you've had a lot of the time-consuming questions already, and don't be alarmed if you see you only have 90-100 seconds per question left. Verbal pacing is a tricky subject and how you should do it depends on how long each question type takes and how much time you typically need to save, but if you finished this test 6 minutes early, it sounds like you can afford to slow down and you probably won't need to worry about pacing too much. Good luck!

Thanks Bunuel souvik101990 bb for the effort. I feel confident now. I have my exam coming up in couple of days and I can focus more on my weak areas and can easily skip through few questions without wasting a ton of time. I hope that the scores I got from gmatprep software was realistic since I tried mimicking the actual test experience. I hope all goes well.

Hi GMATking94,

If you weren't already familiar with those questions, the scores generated by the official practice tests should be representative.

All the best.

This is an AMAZING thread - thanks to Bunuel for initiating it.

I am really keen to know if the same trends continue to hold true for the Focus Edition.

Any data or insights (pun intended 😊) on that would be greatly appreciated!!!!