Last visit was: 23 Apr 2024, 23:38 It is currently 23 Apr 2024, 23:38

Close
GMAT Club Daily Prep
Thank you for using the timer - this advanced tool can estimate your performance and suggest more practice questions. We have subscribed you to Daily Prep Questions via email.

Customized
for You

we will pick new questions that match your level based on your Timer History

Track
Your Progress

every week, we’ll send you an estimated GMAT score based on your performance

Practice
Pays

we will pick new questions that match your level based on your Timer History
Not interested in getting valuable practice questions and articles delivered to your email? No problem, unsubscribe here.
Close
Request Expert Reply
Confirm Cancel
SORT BY:
Date
Math Expert
Joined: 02 Sep 2009
Posts: 92883
Own Kudos [?]: 618653 [1645]
Given Kudos: 81563
Send PM
User avatar
Senior Manager
Senior Manager
Joined: 23 Dec 2010
Status:Making every effort to create original content for you!!
Posts: 442
Own Kudos [?]: 5415 [430]
Given Kudos: 82
Location: United States
Concentration: Healthcare, Social Entrepreneurship
GMAT 1: 660 Q48 V34
GMAT 2: 750 Q49 V42
Send PM
User avatar
Manager
Manager
Joined: 22 Nov 2010
Posts: 202
Own Kudos [?]: 497 [7]
Given Kudos: 75
Location: India
GMAT 1: 670 Q49 V33
WE:Consulting (Telecommunications)
Send PM
Board of Directors
Joined: 01 Sep 2010
Posts: 4380
Own Kudos [?]: 32864 [8]
Given Kudos: 4453
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
5
Kudos
3
Bookmarks
another scenario could be 2 right one wrong (the third) regardless the difficulty and the question.

A sort of a balance approach. I think is woth to taste

best regards

carcass
Math Expert
Joined: 02 Sep 2009
Posts: 92883
Own Kudos [?]: 618653 [15]
Given Kudos: 81563
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
10
Kudos
5
Bookmarks
Expert Reply
3rd scenario added: first 14 questions answered correctly, then 10 questions answered incorrectly and finally the remaining 13 questions answered correctly.
avatar
Intern
Intern
Joined: 05 Sep 2012
Posts: 42
Own Kudos [?]: 74 [3]
Given Kudos: 17
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
3
Kudos
Awesome initiative!! The use of crystal clear language unties all knots.

Eagerly waiting for the Verbal What-if scenarios... :)

Just a quick Q, does GMATPrep simulate experimental questions in the same way as the test???

I think you might have even conducted multiple cycles of the same scenario. Was there no difference in the end scores?

Because I guess only experimental questions can introduce an error margin and it would be great to quantify that if possible... :-D

Thanks to all... :-D
Math Expert
Joined: 02 Sep 2009
Posts: 92883
Own Kudos [?]: 618653 [4]
Given Kudos: 81563
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
3
Kudos
1
Bookmarks
Expert Reply
soumens wrote:
Awesome initiative!! The use of crystal clear language unties all knots.

Eagerly waiting for the Verbal What-if scenarios... :)

Just a quick Q, does GMATPrep simulate experimental questions in the same way as the test???

I think you might have even conducted multiple cycles of the same scenario. Was there no difference in the end scores?

Because I guess only experimental questions can introduce an error margin and it would be great to quantify that if possible... :-D

Thanks to all... :-D


Important Clarification: we have a strong reason to believe that even though GMAT Prep is the closest algorithm to the GMAT, it is most likely NOT identical. Some of these scenarios may result into a different outcome when attempted on the real test. We so far have no reports to confirm or dismiss these results based on the test day experience.
Board of Directors
Joined: 01 Sep 2010
Posts: 4380
Own Kudos [?]: 32864 [6]
Given Kudos: 4453
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
6
Kudos
:roll:
Attachments

screenshot.4.jpg
screenshot.4.jpg [ 74.85 KiB | Viewed 51244 times ]

Founder
Founder
Joined: 04 Dec 2002
Posts: 37301
Own Kudos [?]: 72862 [4]
Given Kudos: 18857
Location: United States (WA)
GMAT 1: 750 Q49 V42
GPA: 3
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
4
Kudos
Expert Reply
GMAC Continues to emphasize that the first 10 questions are not important.... which I think is a very absolute statement which does not appear so absolute based on our observations.
https://www.mba.com/the-gmat/prepare-for ... -that.aspx
Manager
Manager
Joined: 07 Feb 2011
Posts: 79
Own Kudos [?]: 288 [0]
Given Kudos: 47
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
Ok here I go, I hope this doesn't get taken down due to being a childish quesiton but:

about CR placement and its affect on score? Most of my wrong in the Verbal Section are CR and they seem pretty darn insignificant. I am still hitting mid 80s percentile in verbal while getting the majority of CR questions wrong

I'm also confused on how to devise a pacing strategy for verbal as I had done with Math by the correctness/guessing based on the number of question.

and I give me corresponding 15minutes OR 8 questions for quant a couple of minutes plus or minus depending on wher ethey fall into that general pace.

For verbal as of now I mentally just give earlier questions some extra time and pace myself through the rest. It seems like verbal is much less rigid when it comes to results and pacing.
How would I devise a pacing strategy for verbal then since, from what I understand, the placement of the SC/RC is random and more important is what comes first?
GMAT Club Verbal Expert
Joined: 13 Aug 2009
Status: GMAT/GRE/LSAT tutors
Posts: 6917
Own Kudos [?]: 63649 [1]
Given Kudos: 1773
Location: United States (CO)
GMAT 1: 780 Q51 V46
GMAT 2: 800 Q51 V51
GRE 1: Q170 V170

GRE 2: Q170 V170
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
1
Kudos
Expert Reply
Manimgoindowndown, I don't think it's true that CR is insignificant at all, and it's not weighted any differently than the other two verbal question types. If you're doing well on SC and RC, then it's certainly possible to get into the high 30s while having some serious CR deficiencies. But it's not because CR is insignificant--it's just because your overall, combined verbal level merits that score.

And hey, if you're able to score around the 85th percentile with some CR weaknesses, imagine what you could do if you improved your CR skills!

I don't think that pacing benchmarks are usually very helpful on the verbal section. This might sound really obvious, but most verbal mistakes happen because you misread or misinterpreted something, for one reason or another. A very tiny lapse in focus can cause you to miss a question, and that's a (presumably deliberate) design feature of GMAT verbal questions.

I think that benchmarking can actually cause you to make more of those small reading errors on the GMAT. On CR and RC questions, a huge proportion of your time will be spent reading and digesting the passage. You can't really do much to accelerate that process, unless you're willing to sacrifice accuracy. And the same is true when you start looking through the answer choices--what are you going to do, skim them to save time? That doesn't make much sense--if you're not reading carefully, you're extremely likely to miss the question, and you'll only save yourself 10-20 seconds in the process, since the bulk of your time is inevitably spent reading the passage, anyway.

So I just don't see the benefit of benchmarking for most people. If you hurry on verbal questions so that you have time to answer question #41, you'll probably screw yourself out of at least one question early in the test, and that's a crappy tradeoff. If you miss an easy question early, it will change the trajectory of the entire test, and your score will be disproportionately damaged. If you kick butt on questions #1-38, then questions #39-41 won't matter all that much.

I know that I'm in the minority on this, but I think you're better off answering questions thoroughly, methodically, and consistently throughout the verbal section. Don't cheat yourself out of a right answer early in the section just to hit some timing benchmarks.
GMAT Club Verbal Expert
Joined: 13 Aug 2009
Status: GMAT/GRE/LSAT tutors
Posts: 6917
Own Kudos [?]: 63649 [0]
Given Kudos: 1773
Location: United States (CO)
GMAT 1: 780 Q51 V46
GMAT 2: 800 Q51 V51
GRE 1: Q170 V170

GRE 2: Q170 V170
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
Expert Reply
Awesome work, Bunuel and Vercules!

It's interesting to see that omitting the last seven quant questions is so deadly on GMATPrep, since the fine folks at GMAC insist that there isn't a huge difference between guessing and omitting questions at the end of each section. They published an official blog post on that issue a couple of years ago (https://officialgmat.mba.com/2009/09/17/ ... g/#more-57) and there's also an accompanying research study on the GMAC website (https://www.gmac.com/~/media/Files/gmac/ ... ssWhat.pdf).

Basically, GMAC argues that there isn't really a meaningful difference between guessing and omitting the last five questions on verbal. They also argue that you're better off omitting the last five if you're doing badly on the quant section, and you're better off guessing only if you're a "high ability" test-taker on the quant section. Interesting.

I guess there are a few ways to look at the differences between the GMATPrep results and the GMAC's official statements. It's possible that the GMATPrep software runs a substantially different algorithm than the actual test, but I think it's more likely that the dramatic result in Bunuel's study is the result of an extra couple of questions (omitting seven questions probably does substantially more damage than missing five) and the fact that he is clearly in the "high ability" category. :-D

A large proportion of GMAT Club members will ultimately do well on quant, so it's best for most members to guess at the end of the quant section instead of omitting questions. But on the verbal section--or for test-takers who aren't strong at quant--maybe it doesn't really matter all that much?
avatar
Intern
Intern
Joined: 15 Apr 2013
Posts: 8
Own Kudos [?]: 1 [1]
Given Kudos: 2
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
1
Kudos
But is the scoring algorithm here indicative of the actual GMAT?

Also, because we don't know for sure whether we got a question correct or not immediately after answering it, does one dare to actually skip (guess) questions 11-19, and only start trying from questions 20-28... and then guessing the remainder? What if you thought you got them right... but you didn't... now you blew around 1/2 the questions that you could've used to bring your score back up? thoughts?

Also, just wondering how difficult were the 1-10 and 20-28 questions you were getting? were they mostly 700-800 questions??

Either way thanks for your time... its great information but I'm just wondering if I dare to use it as an actual strategy on test day.
Founder
Founder
Joined: 04 Dec 2002
Posts: 37301
Own Kudos [?]: 72862 [0]
Given Kudos: 18857
Location: United States (WA)
GMAT 1: 750 Q49 V42
GPA: 3
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
Expert Reply
simonj13 wrote:
But is the scoring algorithm here indicative of the actual GMAT?

Also, because we don't know for sure whether we got a question correct or not immediately after answering it, does one dare to actually skip (guess) questions 11-19, and only start trying from questions 20-28... and then guessing the remainder? What if you thought you got them right... but you didn't... now you blew around 1/2 the questions that you could've used to bring your score back up? thoughts?

Also, just wondering how difficult were the 1-10 and 20-28 questions you were getting? were they mostly 700-800 questions??

Either way thanks for your time... its great information but I'm just wondering if I dare to use it as an actual strategy on test day.


WE DO NOT KNOW THE ACTUAL GMAT ALGORITHM.
It is in some ways and it is not in others. The official algorithm is much more complex and has multiple checks/verifications to prevent fraud/cheating.
If you have to skip as many questions, it is a tough situation. Try to get a better handle on your timing.
User avatar
Intern
Intern
Joined: 11 Nov 2012
Posts: 10
Own Kudos [?]: 10 [0]
Given Kudos: 38
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
Hi Bunuel,

May I ask which version of the GMAT Prep Software did you use for this analysis?
And how did you reset the software? Because in v2.2.306 we have to login using our mba.com account so I am not sure whether the reset methods given in another post still work.

Thanks.
Math Expert
Joined: 02 Sep 2009
Posts: 92883
Own Kudos [?]: 618653 [0]
Given Kudos: 81563
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
Expert Reply
aquax wrote:
Hi Bunuel,

May I ask which version of the GMAT Prep Software did you use for this analysis?
And how did you reset the software? Because in v2.2.306 we have to login using our mba.com account so I am not sure whether the reset methods given in another post still work.

Thanks.


2.1.294 version.

In that version (at least) there is a reset button under "Take a practice exam" page.
GMAT Tutor
Joined: 24 Jun 2008
Posts: 4128
Own Kudos [?]: 9240 [0]
Given Kudos: 91
 Q51  V47
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
Expert Reply
These kinds of GMATPrep experiments are fine, but it's extremely easy to arrive at incorrect conclusions from them, unless you understand how the scoring algorithm works. Some of the conclusions drawn in this thread are not correct, though in some cases it will take me some time to explain why. I'll try to do so over a few posts in the coming days. For example, here:

Vercules wrote:

Analysis/ Conclusion :



Results from 15 tests show that on average an Option 'C' carries a higher probability of being correct and option B carries the lowest. ...

But if you have to guess one or two and you have no idea/ time for the question mark 'C'.


There is a GMAC research report which explicitly states that they control for 'answer position' when designing the GMAT. That is, no one answer choice is right more often than another. I've also verified that independently; looking over more than 1000 official questions, each answer choice is correct roughly equally often.

That said, if you look only at a small sample of questions, one answer choice might appear to be correct more often than another, just because of random variance. If you do a binomial probability calculation, then if each answer choice has a 20% chance to be correct on a random GMAT question, it would not be all that improbable for one answer choice to be correct 13 times on one GMAT Quant test (that should happen on about 2% of tests). Since your samples do not appear to be independent (I assume you were using the same GMATPrep test repeatedly, in which case it would not be surprising to get similar results each time because of question repetition), the most likely explanation for your findings is pure random luck.

But if instead you assume that your finding is meaningful, and that, when guessing randomly at every question, C is the most likely answer to be correct, what would that finding mean? It might seem paradoxical, but it means that test takers who need to guess almost certainly should not guess C. If you accept (and all the evidence I have confirms this) that each answer is correct about 20% of the time, then if C is the right answer more often when you guess at every question, that means C is more often the right answer on the absolute easiest questions. For C then to be correct 20% of the time overall, C would need to be correct less often on the harder questions. And those are the questions people need to guess at.

Even if you don't believe each answer is right 20% of the time, knowing which answer is more often correct on 200-level questions is of no help to test takers unless they'll need to guess at 200-level questions, which almost no one needs to do. So it would not be a relevant finding for most test takers anyway. But all of the evidence I've seen suggests to me that no one answer choice is a better guess than another, if you need to guess purely at random. The best strategy in that case is simply to guess as quickly as possible.
GMAT Tutor
Joined: 24 Jun 2008
Posts: 4128
Own Kudos [?]: 9240 [2]
Given Kudos: 91
 Q51  V47
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
2
Kudos
Expert Reply
To understand the meaning of other experiments in this thread, it's important to understand how the scoring algorithm works. There are really two parts to the algorithm:

• the actual scoring algorithm, which determines your score based on your answers
• the question selection algorithm, which determines, based on your previous answers, which question you see next

The scoring algorithm itself is completely blind to question position. If you were to take a GMAT, and all 37 questions were 500-level, you'd get exactly the same score by answering the first 25 correctly and the last 12 incorrectly as you'd get by answering the first 12 incorrectly and the last 25 correctly. The scoring algorithm itself knows nothing about where each question was in the test; it's simply not a factor in the calculation.

What does change, though, based on your answers, are the questions you see later in the test. When you do well early on, you'll more likely see hard questions later, and if you do badly early on, you'll more likely see easy questions. Note though that every test is different. If you answer, say, your first five questions correctly (which is outstanding). The test will then be quite confident you're a high-level test taker, and will want to give you a very hard question. But your ability estimate is not the only factor the test uses to select your next question. The test also needs to meet content requirements (more than one - it needs to deliver the correct balance of math topics, and also the correct balance of PS and DS questions), and there are security conditions as well (no question is used too often, in order to prevent people from getting a big advantage by learning about test questions in advance of the test). The question database is not enormous for a single test, which means that, if you get your first 5 questions right, while the test might want to make your sixth question a 760-level question, there may not be a question exactly at that level that meets all of the conditions the test needs to meet. So depending on the database on the day you take your test, you might get a 700-level question at that point, or an 800-level question.

So every test is different, which is why you'll get different results each time when you carry out experiments where you get questions in certain positions wrong (every odd numbered question, say). I have the impression that the assumption behind those experiments is that questions are 'weighted' by their position in the test (that early questions count more than later ones), and that's not the case.

To demonstrate this, I did just carry out two trials using two different GMATPrep Quant tests, where I answered every prime numbered question (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37) incorrectly. On the first attempt, my score was Q33, and on the second, it was Q38. That's quite a big difference.

I'm not precisely sure what the purpose of these experiments is, testing how wrong answers in different places will affect your score, but if the goal is to find certain question numbers that are especially important or unimportant, you simply won't be able to do that. It's not how the test works.
Founder
Founder
Joined: 04 Dec 2002
Posts: 37301
Own Kudos [?]: 72862 [0]
Given Kudos: 18857
Location: United States (WA)
GMAT 1: 750 Q49 V42
GPA: 3
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
Expert Reply
A surprising turn of events. In the Official GMAT Monthly this was featured in their FAQ section:

Q: Are the practice GMAT tests on the software indicative of how well you'll do on the actual GMAT?
A: GMATPrep® software uses real GMAT test questions and the actual GMAT CAT® test delivery and scoring algorithm. It should help you to gauge your preparedness for the GMAT exam. Note - the predictive value of these tests may be affected by the extent to which actual testing conditions are adhered to during practice testing sessions. For example, if you were to suspend testing and resume the test later, an option that you have with GMATPrep software but not with the actual GMAT CAT exam, you might score higher on GMATPrep test than you would otherwise.


Previously it was stated/believed that the GMAT Prep and GMAT used different algorithms (a more simplified algorithm was used in GMAT Prep)
GMAT Tutor
Joined: 24 Jun 2008
Posts: 4128
Own Kudos [?]: 9240 [0]
Given Kudos: 91
 Q51  V47
Send PM
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
Expert Reply
bb wrote:

Previously it was stated/believed that the GMAT Prep and GMAT used different algorithms (a more simplified algorithm was used in GMAT Prep)


I'm almost certain it is not true, and never has been true, that GMATPrep used a different basic scoring algorithm from the real test. I've never heard any official statement that would make me think otherwise, and if there has been one, I'd love to see a link.

The fundamental scoring algorithm used in GMATPrep really has to be the same as is used in the actual GMAT, because the scoring is just based on probability theory, and GMAC isn't free to change the laws of math. There are differences between GMATPrep and the GMAT (for example, GMATPrep does not insert experimental questions in the way the real test does), but those differences don't have to do with the actual scoring of the test.Those differences almost certainly still exist, so I don't think this recent announcement is new information, or signals any change in the GMATPrep tests.
GMAT Club Bot
Re: GMAT Prep Software Analysis and What If Scenarios [#permalink]
 1   2   3   4   5   

Powered by phpBB © phpBB Group | Emoji artwork provided by EmojiOne