GMAT Prep Software Analysis and What If Scenarios

There are exactly 9 (not 13) experimental questions on Quant out of the 37 total questions. 28 are counted. If you happen to get really, really lucky, then maybe 9 of the 12 of the questions you get wrong are experimental. That still means that you will get 3 counted questions wrong, which could perhaps be enough for a Q50 but definitely not enough for a Q51.

Hi, Folks!

I just tested the 7th scenario which is described in the very first post of this thread (Quant: 1-10 correct, 11-19 guessing, 20-28 correct and 29-37 guessing) and I ended up in having Q47.

However, I failed to answer the all the first 10 questions right. (I answered 5,6,8,9 incorrectly).

Guessed 3 questions correctly.

Has anyone tested this strategy on the real exam? This strategy seems too easy for me since I am not really aiming for a 700+ score. Q47 would be more than sufficient for my desired total score.

It's probably impossible to implement such a strategy on the GMAT exam. Even if a test taker who could control the distribution of correct answers wanted to get some of the questions wrong, he or she would have no way to take the effect of experimental questions into account (approximately 24% of all quant questions are experimental).

bb Vercules carcass
is this analysis applicable to the new GMAT exam version too? that is without experimental questions? that is with 36 questions in verbal?

We have not run in experiments with the new GMAT Prep software yet. We will soon but meanwhile you can assume that it’s probably fairly similar or bastard different. I can’t prove either at the moment

Posted from my mobile device

We can fairly assume that it would be similar but it solely depends on the elimination of experimental questions !
all the scenarios for both verbal and quant took into account the experimental questions. so my question is whether the incorrectly attempted questiobs had some or many or none experimental questions? if the incorrect questions had no exp questions then the assumption of the analysis being similar is fairly substantiated but if the incorrect questions had exp questions too then we cannot conclude with any surety . So my question to you is whether there were exp questions in those incorrect choices? or was it possible to know which question was experimental?

The algorithms of gmat prep and official gmat are different. There are supposedly no experimental questions on gmat prep but the system somehow compensated and accounts for that. Thus your question has no answer.

Posted from my mobile device

It is worth noting that GMAC is now stating on its website that "The practice exams use the same scoring algorithm as the official GMAT® exam."

On the old version of the GMATPrep desktop software, the scoring algorithm was NOT the same as that of the real GMAT, in part because the real GMAT used to have 23 experimental questions, and on the GMATPrep desktop software, every question counted.

Yet, GMAC is now telling us the opposite: that the new online tests use the same scoring algorithm as the real GMAT. However, unless there are also 12 questions on the GMATPrep online tests that are experimental/pretest/unscored, this is not completely true.


Source: GMAC Website

Theoretically, however, there is nothing stopping GMAC from throwing 12 experimentals into the GMATPrep software as well--especially since the software is now online and can be updated at any time. Until we run some controlled, detailed simulations with the new online tests, there is no way to know whether some of the questions count, and some don't. If there truly are 12 unscored questions, then maybe the new GMATPrep software (online version) really DOES now have the same scoring algorithm as the real GMAT.

Or maybe it's just marketing hype. Time will tell...

-Brian

Thank you for reporting this!!!

PS. I find it very odd that they managed to copy the algorithm but failed to copy the user interface.

Posted from my mobile device

First, I have to confirm that the score algorithms in GMATPrep (old or new one) and the real test are the same. The only different is because of the input to the adaptive system to calculate score. For example:

• In GMATPrep, there are no experimental questions but there are some experimental questions.
• The question bank in GMATPrep is far less than the question bank in the real test. (So there are some significant results between those flatforms)
• The statistic information for each question is updated regularly in the real test, but not in GMATPrep (old or new one). ie a question could change the difficulty from hard to easy because of the improvement of test takers over time. However, this event will never happen in GMATPrep because GMAC no longer update statistic information of those questions in GMATPrep.

After all, there is no difference in the heart of the score algorithm. GMAC's claim is totally true.

However, adaptive system is really complex and we don't know what the exact result will be since the overall score depends on variety of reasons even that the score algorithm remains unchanged. Let's put it in this way: GMAT is the exam that wants to test the real abilities of test takers. Thus GMAT requires that those abilities must be consistent. Thus, the more consistent the performance during the test is, the higher score will be.

Ah, there is no way to know which questions are experimental. If anyone knew, I'm sure that GMAC would change the algorithm immediately.

Hi broall

I feel GMAC (Even though they deny it) has changed their scoring algorithm in the new format otherwise Q50 with 1 Wong (That too in last section) isn't possible. In GMAT Prep Mock, 7 wrong is fetching Q50. So, obviously GMAT Mock scorinng algorithm and Real GMAT scoring algorithm is not same.

I don't think that GMAC change its scoring algorithm. The score algorithm doesn't care about the number of question answered correctly. It cares about the difficulty of each question.

For example, theoretically, we could still earn V50 if we just answer 50% questions correctly. In the real life, this case will never happen because the question bank doesn't have enough very hard questions for those test takers. Thus, could could see that, in ESR file of Ron Purewal, who has scored perfect 800, the difficulty comes down. The difficulty should go up right?

So, you can't simply conclude that the score algorithm has changed.

P/S: I think that GMAC does make some changes, but that's called system tuning (or calibration). This one shouldn't be called big changes in the core of system.

Hi broall,

As I understand GMAC scoring algorithm (Please correct me if I am wrong) - as you answer the questions correctly the difficulty level goes up. I can't have 7 consecutive answers wrong in First 10 questions and then the GMAC throwing 8th question as 700 Level question and evaluating only the 8th Question correct answer for Scoring.

Sorry, but I dont get your question. Could you explain it?

In the previous post you said that The score algorithm doesn't care about the number of question answered correctly. It cares about the difficulty of each question.

As I understand GMAC scoring algorithm (Please correct me if I am wrong) - as you answer the questions correctly the difficulty level goes up. Suppose, in the first 10 questions, I answer the first 5 questions incorrectly. Will I get 6th question as 700 Level question?? I guess NO. And Suppose, by any chance if get 6th question as 700-Level question and I answer it correctly, Will the system evaluate only 6th question for scoring??

[Suppose, in the first 10 questions, I answer the first 5 questions incorrectly. Will I get 6th question as 700 Level question??]
Of course No. Your guess is correct. Why? Because the system determines that your real ability is low (ie 500-600), so why should it get you a 700 question?

[And Suppose, by any chance if get 6th question as 700-Level question and I answer it correctly, Will the system evaluate only 6th question for scoring??]
No. LOL, why on earth do you think that the system counts only on those questions you answered correctly? It counts all questions and generate the overall score.

The simple answer here is that even when you answer 100% of questions correctly, the question difficulty level does not continue relentlessly upward in a straight line. The test also has other concerns, like feeding you questions of every type/sub-category, and serving you about 15% experimental questions, whose difficulty levels are set in advance.

Also, GMAC is notorious for understating the difficulty level of the questions on the Verbal section. As you noted, even perfect V51 scorers don't see the overall difficulty level go very high, but we know this not to be the case--there are in fact some very difficult questions mixed in there, but the blue line in the ESR only reflects the average difficulty of all questions in that quarter.



-Brian