Serious question about GMAT algorithm

Question

Hi Guys,

I've been thinking lately about GMAT algorithm and came up with one very important question that is being neglected to my surprise.

Does GMAC adaptive algorithm calibrate the difficulty of the questions based on the answers in each sub-group, or just takes into consideration overall performance?

To put it in other words: If I miss the first 8 RC questions deliberately, but Answer the first 8 questions in both SC and CR correctly, will the algorithm take into consideration my skill levels in particular sections and give me EASY RC question and Difficult SC and CR, or it will assume my general Verbal level is medium and give me medium questions in all the 3 modules?

Thanks, y'all
 bb

bb · Answer

There have been quite a few posts about GMAT algorithm. One important point right up front that people often forget is that the algorithm is not disclosed. We all know the concept of it and we all know how it is supposed to work based on the GMAC’s feedback but nobody knows the actual ins and outs. That means well many people agree about certain parts and elements of the algorithm, nobody can know for sure. Thus many of the debates and arguments about more technical aspects of it often remain contested or questioned simply because there’s no authority to confirm it. We can only use logical deduction and run various iterations such as GMAT prep what if scenarios.

It’s to your question, most believe the way the test works, Your performance score is calculated every time you answer a question. Whether that’s based on the traditional GMAT scale or the raw scores or some other methodology, you are constantly moving up or down the scale. The only exception obviously being the experimental questions.

I have never heard a theory about RC, SC, CR being evaluated separately. I don’t think there’s much evidence of support for it out there and it would create more complexity for the test writers. Imagine you would have to marry the three scores together at the end. I don’t see a reason for them to do that.

IanStewart  knows infinitely more about the algorithm than I do. I tend to follow his school of thought. 😎

Posted from my mobile device

dato10kokli · Answer

Thanks for your prompt answer  bb

Even pentagon’s secrets are leaked here and there so its very strange none of the insiders disclosed some of the details given the large timespan gmac has been around

Posted from my mobile device

bb · Answer

I think in part the the algorithm is straightforward enough and so people would be disappointed by the leak

The algorithm is based on 1950s and 60s research and it doesn’t appear to be anything fancy besides some additional layers of security and cheating validations As well as improvements that allow GMAC to conduct Tests using fewer questions. Some of the questions are designed so they would use up time through mechanics rather than difficulty. That still makes them harder because it takes longer to solve them. These are more of filler questions that seem to come in the second half of the test... just my observation. Of course it’s only a theory. Posted from my mobile device

IanStewart · Answer

Verbal adapts by question, not by question type. If you're doing amazingly at CR an SC, but terribly at RC, then you'll be seeing mostly medium and medium-hard questions, whether those are RC questions or CR/SC questions. RC though doesn't adapt strictly by question - each passage has a fixed batch of questions attached to it. So if the test thinks you're above average, and wants to give you a medium-hard RC passage, some of the individual questions attached to that passage might be easy, and some might be very hard, and they'll 'average out' to a medium-hard batch of questions.

My recollection of this is faint, so I might be wrong on the details, but as I recall: a few years ago, one of GMAC's official representatives on one of the main GMAT forums said, responding to a question just like yours, that CR, SC and RC do adapt independently. So you might find some seemingly official replies somewhere that say exactly the opposite of what I've just said in the above paragraph. But that same rep later corrected that information -- it was wrong.

dato10kokli · Answer

bb 
 IanStewart

Maybe that's just wishful thinking on my part, I was very weak at SC and have great gains thanks to  GMATNinja

So if what you tell me is correct, that's bad news for me, improving SC skills will mean harder questions in other modules as well to me, so I might not be that good at RC and CR as I thought before.

Thanks again

bb · Answer

I don’t think it is ever good to be lopsided. You probably don’t want to have a massive right arm and a teeny fragile body

But it would be weird if you we’re constantly going back-and-forth between SC and CR going between super easy and super hard. I guess the ultimate proof for you that I can offer is the enhanced score reports. You can see than certain people get low grades or low percent of questions correct for SC or CR but you can also see the average difficulty of questions that tends to be like a straight line rather than a zigzag. I’m afraid your assumption is wrong about how the algorithm scores and serves questions. At the same time, getting harder questions wrong gives you less penalty than getting easier questions wrong. The way to get a high score on the GMAT is to solve hard questions and answer them correctly so I don’t think you should quite lament your sentence correction skills. It is going to help rather than hurt. I don’t think you wanna sabotage your sentence correction in other words. So technically that makes both of your assumptions off 😂😂😂 Posted from my mobile device

dato10kokli · Answer

bb

Yep SC improvement will help me but the impact will be less than it would be if my wishful thinking were true.

While missing 6-7 questions in the SC section (mostly in the first part of the sentence), I barely missed any RC or CR question, so I inferred that I was really pro at RC and CR, but it turns out that my poor SC performance was dragging the difficulty of them down, and after I improve my SC, I will start missing RC and SC points as well (while if my wishful thinking was true, my CR and RC performance would remain the same).

Anyways, I think I am being too frugal with the 4 official tests that I've not taken yet, they will make things clearer.

AjiteshArun · Answer

Hi  dato10kokli ,

I don't think it's bad news. For all you know, you got some very easy questions wrong, and that could be a great opportunity for you. I would certainly look at doing reasonably well across all three question types as being an easier way to get a good score than performing extremely well in RC and CR and doing very poorly in SC.

IanStewart · Answer

I think you might be misunderstanding how the algorithm works, if I'm understanding correctly what you're saying. It sounds as though you think your RC and CR score will fall if you get harder questions, because you'll get more questions wrong. That's not the case. To illustrate how things work, I'll just make up percentages so I don't need to get into technicalities, but if a test taker is at a V40 level in CR, and gets a test only containing CR questions, then if that test contained only V20-level questions, the test taker might get about 95% of them right. If the test contained only V30-level questions, that test taker might get about 80% of them right. And if the test contained only V40-level questions, that test taker might get 60% of them right. Yes, as the questions get harder, the test taker has more wrong answers. But that's exactly what is supposed to happen. The test taker's score does not change.

So if you improve at SC, and as a consequence get harder questions in CR and RC on your test, sure, you'll probably have more wrong answers in CR and RC. But that won't affect your score at all (in fact, your score will improve, because you'll be just as good as before at CR and RC, but will be better at SC).

IanStewart · Answer

And just about one other thing:

It's truly a myth that the algorithm is a secret, and anyone who claims it's "proprietary" or a "secret" is frankly misinformed. The important parts of the algorithm are public knowledge. There are some technical details that are not public knowledge, but they're really not important. Anyone who understands what those details are wouldn't even talk about them being 'leaked', because I can't imagine GMAC would care if they were publicized.

The scoring algorithm is just based on something called a 3PL model, which is described in dozens of places, including on wikipedia. The main part of the scoring algorithm is not something unique to the GMAT; it's used in many applications (I've seen articles about testing airline pilots, and for taking mental health measurements, that describe how to use the same kind of algorithm for those purposes). There's a whole field of academic research that has been around for many decades called Item Response Theory (IRT) that studies tests and assessment generally, and the GMAT uses a system developed by IRT scholars back in the 1960s.

The math involved is pretty complicated (if you don't have a strong foundation in calculus, statistics, and probability theory, there'd be no way to really make sense of it), though the underlying principles are fairly simple. But knowing how the algorithm works really doesn't help. It's a bit like if you take a standard multiple choice test where all that matters is how many right answers you get -- you know how the 'algorithm' works (they'll count up your correct answers) but knowing that 'algorithm' doesn't help you get a better score. What helps you to get a better score is learning how to answer more questions. The same is true with the GMAT - if you take the test in the normal way (i.e. solve the questions you can answer, don't waste too much time on questions you can't), you're basically using the optimal strategy.

It's only if you believe some of the nonsense about the scoring algorithm that you still find in many prep books, and in many posts on this forum, and accordingly adopt a suboptimal strategy, that your 'knowledge' of the algorithm can affect your score. Then your score can suffer. As long as you're using a good pacing strategy, and you're taking care to avoid careless errors on easier questions, there's no knowledge of the scoring algorithm that will magically improve your score beyond your true ability level (despite what many prep companies falsely claim).

dato10kokli · Answer

IanStewart

I am afraid I conveyed my idea in a poor manner, I will clarify my point:

I know that increasing my SC proficiency will result in an increased overall score, even though I might miss additional questions in other modules due to increased difficulty. But this effect would be even greater if My wishful thinking was true and increasing SC proficiency would not lead to increased difficulty in other modules.
Thats all I wanted to say

	Target Test Prep - Save up to $599 $400 Discount + $99 Bonus GMAT Club Tests
	e-GMAT: 60% Off on GMAT Focus Get to 735+ from any starting point
	Magoosh increases your score, guaranteed - 15% off ALL Magoosh GMAT plans with code GMATCLUB15 + Free GMAT Club Tests.
	Manhattan Prep: $99 Bonus Score Higher on Your GMAT Exam. Guaranteed.
	Experts' Global - ~~$1720~~ $960 End-to-End GMAT Prep + App Support: 150+ Videos, 2000+ Questions, 15 FLTs +
	GMATWhiz - $299 ~~$499~~ Experience the course through a free trial.
	GMAT Club Tests - Free Included with every course purchase of $149+ View all deals

Serious question about GMAT algorithm

Prep Toolkit

Top 5 GMAT Debrief Videos

615 to 715 in 15 Days (Ria's Strategies)

More than Quant/Verbal Abilities: Speed vs. Accuracy

745 with Only Self-Prep and GMAT Club

From 555 to 765: 210-point Improvement

Perfect 805 Score Debrief by Julia

My Rewards