Conclusions from GMAT algorithm study based on ESR analyses

This list has been slightly updated. It is pretty much on point

Thanks to GMAT Club members sharing their ESR's, especially in this thread: https://gmatclub.com/forum/information-o ... 21111.html, I now have access to about 40 ESR's. While more reports are definitely needed to help draw robust conclusions, here are a few things that I have noticed and are quite confident about.

1. The distribution of questions by quarter:

-Verbal: 10 (2 exp) - 10 (3 exp) - 10 (3 exp) - 11 (3 exp); 41 questions including 11 experimental
-Quant: 9 (2 exp) - 9 (2 exp) - 9 (2 exp) - 10 (3 exp); 37 questions including 9 experimental

You see, there are A LOT of experimental questions in each section.

2. Sectional weighting

Whether the first 10 questions are more heavily weighted than others has been a subject of debate in our community for a while. I have dug through a lot of the ESR's I have my hands on to draw this important conclusion:

There is no evidence that the first 10 questions are more heavily weighted than the rest of the section. In fact, I think we have nearly sufficient evidence to conclude the opposite: that the first 10 questions are not more heavily weighted.

I have a few reports where the test taker gets several questions wrong in the first quarter but destroys the rest of the section and ends up at the same place as another test taker who scores perfectly in the first quarter but gets more questions wrong later on. It is rare for someone to not do so hot in the first part of the section but then improve significantly in the remainder of the section when the questions tend to get progressively harder if he is doing well. However, it is very common for the opposite scenario to occur. Quite understandably, the more questions you answer correctly, the more difficult the questions tend to get and thus the easier it is to answer subsequent questions incorrectly.

But the bottom line is, don't try too hard in the first quarter at the expense of the rest of the section.

3. Sectional difficulty

This is the conclusion that puzzled me the most:

Sometimes, when you're doing too well, e.g. answering all questions correctly, the difficulty of questions drops.

When you face less difficult questions, each question you answer wrong tends to get more heavily penalized. Perhaps the GMAC wants to make sure you've got all your bases covered so you don't answer hard questions right but easy questions wrong?

4. Sub-sectional scaled score and percentile

Not sure how to post a scaled score vs percentile table without making it look messy, so I'm probably not going to post all percentiles; rather, I'm going to share a few observations I consider interesting:

- Sentence Correction is the hardest sub-section to score perfectly on. A 51 on SC warrants a 99th percentile, whereas the same feat in CR bears a 98th percentile rank, and that in RC would give you a mere 95th percentile.
- Reading Comprehension is the hardest sub-section to score 40+ on. In my sample, there are a few 51's but nothing between 42 and 50.


I apologize for the messy formatting job; this post is a very rough draft and will be continually updated as I have more time for analyses and reformat and as more ESR's become available. If someone that's skilled in formatting can jump in to help with making the content visually appealing, that would save me some time and effort to concentrate on other pieces. :wink:

We need more input. Your contribution to this thread will be super helpful: https://gmatclub.com/forum/information-o ... 21111.html

5. Impact of number of questions wrong on scores

These are the rough ranges for the number of questions people have got wrong to achieve a certain score:

Q51: 0-2
Q50: 1-6
Q49: 4-8
Q48: 6-9
Q47: 7-11
Q46: 9-10
Q45: 10-11

Keep in mind that these are out of the 30 Quant questions that count

V50: 0
V48: 1
V47: 2
V46: 2-3
V44: 3-5
V42: 4-5
V41: 4-6
V40: 7
V38: 8-11
V37: 8-10
V36: 8-10
V35: 9-12
V34: 7-11

Keep in mind that these are out of the 28 Verbal questions that count

HiLine, Could you please modify the above table? I've seen examples of a V41 and V40 with as many as 10 wrong.

The Quant tables need to modified significantly as well. I have seen scores as low as Q42, for example, with only 9 questions wrong.

Thanks,
-Brian

Brian, thanks for bringing these examples to my attention. I have indeed seen V40 for 9 questions wrong and have incorporated this data point into the table. The V41 score, isn't it for the new format? I am hesitant to mix the two formats together. As for the Quant part, I decided to set the score cutoff at 45 because that is already at the 60th percentile with about 10 wrong questions. Once we go down past this point the variation is so great that it is a futile exercise to report all the possibilities.

Sure, I understand not wanting to go below Q45 for practical reasons, but if you can score Q42 with only 9 questions wrong, then the low cutoff for Q45 must be even lower than currently indicated (6 or 7 wrong, for example, instead of the listed 10).

I understand that this is just conjecture until we collect more ESR data. However, in general it's safe to assume that the lower the score, the wider the range of potential questions wrong, which suggests that the ranges should be getting larger (not smaller) as section scores decrease, not the other way around.

In other words, it doesn't make sense for Q50 to have a wide range of 1-6 questions wrong, for example, while Q45 would have a narrow range of only 10-11 questions wrong. I would guess that in reality the range of Q45 is more like 6-12 questions wrong.

-Brian

Hi mcelroytutoring

Can you please tell me what is the number of Question that could go wrong to get a 50-51 in Quant and 41-42 in Verbal in the new GMAT format.

Thanks!

5. Impact of number of questions wrong on scores

These are the rough ranges for the number of questions people have got wrong to achieve a certain score:

Q51: 0-2
Q50: 1-6
Q49: 4-8
Q48: 6-9
Q47: 7-11
Q46: 9-10
Q45: 10-11

Keep in mind that these are out of the 30 Quant questions that count

V50: 0
V48: 1
V47: 2
V46: 2-3
V44: 3-5
V42: 4-5
V41: 4-6
V40: 7-10
V38: 8-11
V37: 8-10
V36: 8-10
V35: 9-12
V34: 7-11

Keep in mind that these are out of the 28 Verbal questions that count

Hi! Could you please tell me whether this analysis is still valid (after the GMAT has changed in April 2018).
Shouldn't the two statements be flipped? I.e. Verbal should have 30 questions that count (instead of the above stated 28) and Quant should have 28 questions that count (instead of the above stated 30).
Correct me if Im wrong!

Hi! Could you please tell me whether this analysis is still valid (after the GMAT has changed in April 2018).
Shouldn't the two statements be flipped? I.e. Verbal should have 30 questions that count (instead of the above stated 28) and Quant should have 28 questions that count (instead of the above stated 30).
Correct me if Im wrong!

Hi GMAT Club experts,

I recently appeared for the GMAT and scored a 720 (Q50 V37). I got my hands on the ESR today and it seems I got only 5 verbal questions incorrect and still ended up with a V37. It seems to be very far off from what has HiLine compiled. Any idea why this might occur? Happy to share my ESR if it helps.

https://i.imgur.com/Clyg9yk.png

But the 48 misses 2 of the last 14. And the the 44 misses 7 of the last 14, at pretty similar overall average levels.

So, again, we see a similar beginning (arguably a BETTER beginning) resulting in a lower score based on performance of the end. I think this furth punctures the idea that the 'first questions matter more.'