ESR Doesn't seem to add up

This is definitely true, and it's fairly easy to see, intuitively, why it would be incorrect to average test subscores to arrive at an overall score. If a test has two sections, and those sections are independent (a test taker's skill at one is independent of their skill at the other), and a test taker gets a 90th percentile score in each section, you might, at first, think that test taker deserves a 90th percentile score overall, when you combine the two section scores. But this 90th/90th percentile score combination is extremely rare. Only 10% of test takers did better on the first section, and only 10% did better on the second. If the sections are independent, then, multiplying probabilities, only 1% of test takers did better on both sections. Now, some test takers outside of that 1% would also deserve a higher overall score than the 90th/90th percentile test taker (someone with a 88th/98th percentile split, for example) but this just illustrates that two high scores are much less likely than just one, and when a test taker has 90th/90th percentile scores, their total score should be much higher than the 90th percentile. The mathematically correct way to combine scores in this situation is to "sum two normal random variables" (something you'd learn about in an undergraduate statistics course), and if you do that with two 90th percentile scores, you end up with a 96.5th percentile score overall.



No, we do know for sure, and in the original thread about that V50 score, which is here, I explained in a lot of detail why it is mathematically impossible for an adaptive test to use experimental questions to calculate a score. I'll summarize here: if a question is experimental, not only does the algorithm not yet know the question difficulty, discrimination, or guessing parameter values. so the algorithm would have no idea what a right or wrong answer even meant, it doesn't even know if the question is a valid test question. Invalid questions give false information about a test taker if they're used on a test, so experimental questions can never and would never be used as part of the score calculation.

There's a different explanation for that V50 score when the test taker answered every Verbal question correctly, and the associated ESR illustrates what happened (the ESR can be found in the above thread) : by fluke, that test taker had a very easy Verbal test on average. That can occasionally happen, because the GMAT does not adapt nearly as predictably as most prep books claim. And if a test taker gets everything right, all the algorithm can really say is "the test taker is above level X". If a test taker only gets V30-level questions, and answers them all correctly -- well that's what a V50 test taker will usually do, and is also what a V51 test taker will usually do. The algorithm can only decide between V50 and V51 by asking "which is more likely?" and V50 is the test taker's more likely level, because more people are V50-level than are V-51 level.

Of course, it wasn't the test-taker's fault that they didn't get many hard questions -- it was a fault in the question pool (and the algorithm didn't handle this very unusual case correctly) -- which is why GMAC revised the test taker's score to a V51 on appeal.

Even if someone disbelieves everything I've said on this topic, and thinks "this V50 score report is evidence that GMAC uses experimental questions to calculate test scores", there is one obvious question that needs an answer: if that V50 was the test taker's correct score because the test taker had a wrong answer to an experimental question, then why did GMAC revise the score to a V51?

As for ZS24's questions, it's really hard to say anything precise without seeing the relevant ESR, but the distinctions between high Verbal scores are very fine (one wrong answer can make quite a difference).

Hi all -

Appreciate all the feedback provided. While I understand that the average score of RC, CR, and SC does not come out to the composite, my point was that it was clear from the ESR that my worst performance was in RC, despite a seemingly high percentage of questions correct.

I've attached the ESR here so that it may be more clear - and again, thanks so much for the feedback.

I looked at your ESR, and it does look like you had a bit of a strange test. Your test was actually a bit similar to the one I discussed above (the one that the V50/V51 test taker had), in that you did very well, but still didn't see many hard questions. In theory, that shouldn't affect your score -- when a test has mostly easier questions, you need a very good hit rate to get a great score, and when a test has mostly hard questions, you don't need nearly as good a hit rate to get a great score, but the V41 level test taker should get the same score either way. The one issue, when a test doesn't deliver enough questions around your level, is that there's a larger margin of error, so while your performance was probably correct assessed (based on what you did on the test) as a V41, the algorithm was probably less sure you were a V41 than it would have been had it given you more hard questions.

I'd be reasonably confident this is what happened on your test: your first wrong answer was probably RC, your second was probably CR, and your third was almost certainly SC. The wrong answer to the very hard question barely hurt you, but if your RC questions were, on average, a bit lower level than your SC and CR, and you got one of the medium-level questions wrong, your score splits would make sense. When you divide the test up into small samples, as the ESR does when it reports individual RC, CR and SC scores, then each question has an outsize effect on the resulting score, and wrong answers to lower level questions hurt more than most test takers expect.

That said, I wouldn't suggest reading anything into differences as small as you're seeing on your ESR, considering they're based on very small sample sizes. You should treat that V37 in RC as if it has perhaps a 10 point margin of error, and you'll have a lot more evidence than ten questions on one test can provide about your relative strengths at RC, CR and SC. And since this appears to be a lower-confidence V41, if you have evidence from your prep that you usually score better than that (if you had several V45 scores in official practice tests, say), then I'd recommend you just sign up for another test as soon as you can. Your next test is very likely to be normal, in the sense that it's likely to have much harder questions, and then the test will do a better job of figuring out your true level. The only reason you might wait to retake, assuming you'd find it worthwhile to try to improve your score, is if you wanted to lock in a higher Quant score. I'd expect someone with your scoring profile should be able to consistently hit the Q50 level with the right preparation -- you'd want to focus on developing a better conceptual understanding, because it's understanding that the highest level questions are testing, and to push your Q49 to a Q50, you need to get a couple more of the hardest questions right.

Most of the time, three wrong answers in Verbal produces a V43-V45 score (which is the main reason I find your ESR a bit unusual), so if you can maintain your hit rate on a slightly harder test, you'll definitely get a higher score. Good luck!

Agree with this. If you constantly have that verbal accuracy, you'll be fine if you retake.

Hi ZS24,

I agree that the most likely explanation is that the test gave you mostly "easy" questions (GMAT questions aren't really put into categories like "easy", "medium", and "hard").

This is something that seems (to me) to be happening with increasing frequency. Linking to another, possibly similar, post/ESR.

From what I have seen more recently up until this very ESR, -1 question translates to a 48, -2 to a 47, and -3 to a 44-45. The difficulty of the questions did not seem to matter, although I suppose that the type of test-taker who will miss only up to three questions will likely be seeing medium-level to hard-level questions throughout. I cannot speak to exam trends prior to the redesign in 2018 (when the exam sections were shortened), for instance whether a V50 is possible anymore after missing a question, nor can I say that I have scoured the Internet for every last score report I can find.

Everyone seems to agree, in any case, that the thread is aptly titled. By the time you receive a meaningful response from GMAC™, ZS24, if you receive one at all, you will probably want to move on from the exam altogether. I second the notion that IanStewart has proposed above: take the test again once you have put in a bit more work on Quant. If you can miss three Verbal questions in one test, you can probably miss that many or fewer on another, and the low-ceiling-question-difficulty/score anomaly is not likely to repeat.

I would hate to have to pay for the exam again, but if you hit that target score in your next effort, I think you would hardly care. Best of luck, whatever you decide.

- Andrew

Hi ZS24,

Are you sure that you finished the Verbal section, and confirmed the final answer? There is a "significant" penalty for not finishing, according to GMAC, and this could help explain your V41 score with only 3 questions incorrect better than any of these difficulty-level theories.


See this thread for more info:

https://gmatclub.com/forum/v42-with-onl ... l#p2844651