ESR Doesn't seem to add up

Question

Hi all -

I ended up with a V41 on my second attempt at the GMAT but when looking at my ESR today I didn't seem to understand how my score dropped as much as it did.

For RC I answered only one question wrong it looks like (83% correct Identity Inferred Idea, 100% correct on Identity Stated Idea) yet my score dropped to an equivalent of 37 according to the ESR. That coupled with the fact that the easiest question I got wrong was "medium" according to the Score Report is making me question how that's possible.

My SC and CR score equivalents were 45 and 41 respectively, therefore it's clear that RC is what tanked my score a bit. I know it seems nit picky to be asking about this, but the difference between a 730/49Q/41V and a 740/750 would be massive for my profile.

I'd love perspective from the community on this, as I've tried reaching out to GMAC with no response thus far.

bb · Answer

Hi. It is good you’re looking at your ESR. I think it’s good to question things. I also think a picture is worth 1000 words it’s a little bit hard to understand where you’re coming from without seeing it really.

I can also tell you that scoring errors are quite rare on the GMAT. I only know of a single instance open edge case when someone got V50 instead of V51 without making a single mistake on the GMAT verbal.

Posted from my mobile device

AndrewN · Answer

Hello,  ZS24 . I agree with   bb   and think you should have more Experts take a look at your ESR. Are you certain you missed a single RC question, for example? I would guess you made more mistakes. Keep in mind that the ESR only tells you the "Average Difficulty" of questions within each (approximate) quarter of a given section, so you might have missed, say, an Easy and a Medium question or simply gotten more Inference questions than you believe. Furthermore, the scaled score for the Verbal section is not an average of your three sub-scores, although your particular scores might lead you to believe as much. I am looking at five different ESRs as I write this, and in each case but one, the average would yield a different value—curiously enough, the score would be higher three times but lower in one instance. The bottom line: You cannot place too much stock in the "equivalent" score. Neither do scaled scores per section necessarily translate into a particular total score. (A member named  rudywip , I believe, posted a recent debrief on his experience taking the Enhanced Online Exam; despite having earned identical scaled scores as he had in practice, his 730 turned into a 710 on the actual exam.)

I also remember seeing that scaled 50 score in Verbal with no apparent errors, giving rise to the notion within tutoring circles that perhaps those integrated experimental questions really do count. Only GMAC™ knows for sure.

- Andrew

IanStewart · Answer

This is definitely true, and it's fairly easy to see, intuitively, why it would be incorrect to average test subscores to arrive at an overall score. If a test has two sections, and those sections are independent (a test taker's skill at one is independent of their skill at the other), and a test taker gets a 90th percentile score in each section, you might, at first, think that test taker deserves a 90th percentile score overall, when you combine the two section scores. But this 90th/90th percentile score combination is extremely rare. Only 10% of test takers did better on the first section, and only 10% did better on the second. If the sections are independent, then, multiplying probabilities, only 1% of test takers did better on both sections. Now, some test takers outside of that 1% would also deserve a higher overall score than the 90th/90th percentile test taker (someone with a 88th/98th percentile split, for example) but this just illustrates that two high scores are much less likely than just one, and when a test taker has 90th/90th percentile scores, their total score should be much higher than the 90th percentile. The mathematically correct way to combine scores in this situation is to "sum two normal random variables" (something you'd learn about in an undergraduate statistics course), and if you do that with two 90th percentile scores, you end up with a 96.5th percentile score overall.

No, we do know for sure, and in the original thread about that V50 score, which is  here , I explained in a lot of detail why it is mathematically impossible for an adaptive test to use experimental questions to calculate a score. I'll summarize here: if a question is experimental, not only does the algorithm not yet know the question difficulty, discrimination, or guessing parameter values. so the algorithm would have no idea what a right or wrong answer even meant, it doesn't even know if the question is a valid test question. Invalid questions give false information about a test taker if they're used on a test, so experimental questions can never and would never be used as part of the score calculation.

There's a different explanation for that V50 score when the test taker answered every Verbal question correctly, and the associated ESR illustrates what happened (the ESR can be found in the above thread) : by fluke, that test taker had a very easy Verbal test on average. That can occasionally happen, because the GMAT does not adapt nearly as predictably as most prep books claim. And if a test taker gets everything right, all the algorithm can really say is "the test taker is above level X". If a test taker only gets V30-level questions, and answers them all correctly -- well that's what a V50 test taker will usually do, and is also what a V51 test taker will usually do. The algorithm can only decide between V50 and V51 by asking "which is more likely?" and V50 is the test taker's more likely level, because more people are V50-level than are V-51 level.

Of course, it wasn't the test-taker's fault that they didn't get many hard questions -- it was a fault in the question pool (and the algorithm didn't handle this very unusual case correctly) -- which is why GMAC revised the test taker's score to a V51 on appeal.

Even if someone disbelieves everything I've said on this topic, and thinks "this V50 score report is evidence that GMAC uses experimental questions to calculate test scores", there is one obvious question that needs an answer: if that V50 was the test taker's correct score because the test taker had a wrong answer to an experimental question, then why did GMAC revise the score to a V51?

IanStewart · Answer

As for ZS24's questions, it's really hard to say anything precise without seeing the relevant ESR, but the distinctions between high Verbal scores are very fine (one wrong answer can make quite a difference).

WM413 · Answer

Hi all -

Appreciate all the feedback provided. While I understand that the average score of RC, CR, and SC does not come out to the composite, my point was that it was clear from the ESR that my worst performance was in RC, despite a seemingly high percentage of questions correct.

I've attached the ESR here so that it may be more clear - and again, thanks so much for the feedback.

IanStewart · Answer

I looked at your ESR, and it does look like you had a bit of a strange test. Your test was actually a bit similar to the one I discussed above (the one that the V50/V51 test taker had), in that you did very well, but still didn't see many hard questions. In theory, that shouldn't affect your score -- when a test has mostly easier questions, you need a very good hit rate to get a great score, and when a test has mostly hard questions, you don't need nearly as good a hit rate to get a great score, but the V41 level test taker should get the same score either way. The one issue, when a test doesn't deliver enough questions around your level, is that there's a larger margin of error, so while your performance was probably correct assessed (based on what you did on the test) as a V41, the algorithm was probably less sure you were a V41 than it would have been had it given you more hard questions.

I'd be reasonably confident this is what happened on your test: your first wrong answer was probably RC, your second was probably CR, and your third was almost certainly SC. The wrong answer to the very hard question barely hurt you, but if your RC questions were, on average, a bit lower level than your SC and CR, and you got one of the medium-level questions wrong, your score splits would make sense. When you divide the test up into small samples, as the ESR does when it reports individual RC, CR and SC scores, then each question has an outsize effect on the resulting score, and wrong answers to lower level questions hurt more than most test takers expect.

That said, I wouldn't suggest reading anything into differences as small as you're seeing on your ESR, considering they're based on very small sample sizes. You should treat that V37 in RC as if it has perhaps a 10 point margin of error, and you'll have a lot more evidence than ten questions on one test can provide about your relative strengths at RC, CR and SC. And since this appears to be a lower-confidence V41, if you have evidence from your prep that you usually score better than that (if you had several V45 scores in official practice tests, say), then I'd recommend you just sign up for another test as soon as you can. Your next test is very likely to be normal, in the sense that it's likely to have much harder questions, and then the test will do a better job of figuring out your true level. The only reason you might wait to retake, assuming you'd find it worthwhile to try to improve your score, is if you wanted to lock in a higher Quant score. I'd expect someone with your scoring profile should be able to consistently hit the Q50 level with the right preparation -- you'd want to focus on developing a better conceptual understanding, because it's understanding that the highest level questions are testing, and to push your Q49 to a Q50, you need to get a couple more of the hardest questions right.

Most of the time, three wrong answers in Verbal produces a V43-V45 score (which is the main reason I find your ESR a bit unusual), so if you can maintain your hit rate on a slightly harder test, you'll definitely get a higher score. Good luck!

MBAaustralia · Answer

Agree with this. If you constantly have that verbal accuracy, you'll be fine if you retake.

AjiteshArun · Answer

Hi  ZS24 ,

I agree that the most likely explanation is that the test gave you mostly "easy" questions (GMAT questions aren't really put into categories like "easy", "medium", and "hard").

This is something that seems (to me) to be happening with increasing frequency. Linking to  another, possibly similar, post/ESR .

AndrewN · Answer

From what I have seen more recently up until this very ESR, -1 question translates to a 48, -2 to a 47, and -3 to a 44-45. The difficulty of the questions did not seem to matter, although I suppose that the type of test-taker who will miss only up to three questions will likely be seeing medium-level to hard-level questions throughout. I cannot speak to exam trends prior to the redesign in 2018 (when the exam sections were shortened), for instance whether a V50 is possible anymore after missing a question, nor can I say that I have scoured the Internet for every last score report I can find.

Everyone seems to agree, in any case, that the thread is aptly titled. By the time you receive a meaningful response from GMAC™,  ZS24 , if you receive one at all, you will probably want to move on from the exam altogether. I second the notion that  IanStewart  has proposed above: take the test again once you have put in a bit more work on Quant. If you can miss three Verbal questions in one test, you can probably miss that many or fewer on another, and the low-ceiling-question-difficulty/score anomaly is not likely to repeat.

I would hate to have to pay for the exam again, but if you hit that target score in your next effort, I think you would hardly care. Best of luck, whatever you decide.

- Andrew

mcelroytutoring · Answer

Hi ZS24 , Are you sure that you finished the Verbal section, and confirmed the final answer? There is a "significant" penalty for not finishing, according to GMAC, and this could help explain your V41 score with only 3 questions incorrect better than any of these difficulty-level theories. https://www.drivecms.com/uploads/mcelroytutoring.com/Screen%20Shot%202021-08-22%20at%207.14.10%20AM.png See this thread for more info: https://gmatclub.com/forum/v42-with-onl ... l#p2844651

	Magoosh increases your score, guaranteed - 15% off ALL Magoosh GMAT plans with code GMATCLUB15 + Free GMAT Club Tests.
	Manhattan Prep: $79 Bonus Score Higher on Your GMAT Exam. Guaranteed.
	Target Test Prep - Save up to $325 $200 Discount + $79 Bonus GMAT Club Tests
	e-GMAT: 60% Off on GMAT Focus Get to 735+ from any starting point
	Experts' Global - ~~$1720~~ $960 End-to-End GMAT Prep + App Support: 150+ Videos, 2000+ Questions, 15 FLTs +
	GMATWhiz - $299 ~~$499~~ Experience the course through a free trial.
	GMAT Club Tests - Free Included with every course purchase of $149+ View all deals

ESR Doesn't seem to add up

Prep Toolkit

Top 5 GMAT Debrief Videos

615 to 715 in 15 Days (Ria's Strategies)

More than Quant/Verbal Abilities: Speed vs. Accuracy

745 with Only Self-Prep and GMAT Club

From 555 to 765: 210-point Improvement

Perfect 805 Score Debrief by Julia

My Rewards