(Un) Scientific analysis of gmatprep

Wow, great analysis bsd!! +1

I guess it all makes sense. Now, I've been told that for RC, the questions are not adaptative. From what I understood, you are given a text with a set of questions based on the level of SC and CR questions you've answered before the RC (this would explain why you never start off with RC).

But then within the RC, the questions do not change. So whether or not you correctly answer your first RC question, the second question will be the same. This is what I've been told and having done PowerPrep 1 and 2 three times, I think it might be true
What do you all think?

If this is true, it would explain why in your experiment, when you answered only the RC questions correctly, you got a worse score than when answering SC or CR alone.

Keep posting your findings!!

Awesome posts by bsd_lover and suyashjhawar.
kudos to both

But I think the algorithm in real test must be more complicated than what we see on GMATprep.

There is a danger in drawing conclusions from a small sample size. I agree with Kevin that your Quant performance will have no effect at all on the questions you see in the Verbal section- the test would be unfairly biased if it did- and I expect what bsd noticed was just coincidence.

It is also untrue that SC questions are 'worth more' than CR questions, or vice versa, though I do agree that it's easier to improve on SC than on CR, so it is a good area to focus on during preparation for the test. I do suspect virflo is right in saying that RC questions come in batches- with one passage, some questions will be easier than others, though certainly some RC sets will be harder overall than other RC sets. As a consequence, it may be true that some RC questions are less important than others- those that are far from your ability level.

I do not agree at all with bsd's conclusion about the first ten questions, and take issue with the methodology. If you guess at the first 10 questions, and answer 2 correctly, that is an absolutely horrific performance. That's exactly how well you'd expect a ham sandwich to do on the first ten questions, by guessing. Since you're more than a quarter way through the test at that point, you don't have much time to recover. In addition, since only between 23 and 30 questions actually count (because of the presence of diagnostic questions), you may be more than one third into your test after ten questions. In any case, the conclusion is not relevant to any of us, since none of us will answer 8 of the first 10 questions incorrectly, unless we are genuinely sub-300 scorers on the GMAT. Yes, you can conclude from the result that it's a bad idea to guess at the first 10 questions, but I don't think that should come as any surprise. I agree entirely with terp, that the best strategy is to do as well as possible on every question, and it is certainly true that a series of bad answers at the end of a test can absolutely kill your score.

Whoa - just stumbled unto this thread - fantastic job BSD_Lover!

Kudos, bsd! Great, great, great analysis.

I'm not sure if it's out there - Is the GMATPrep scoring algorithm the same as the real GMAT? Does GMAC, ACT, Pearson VUE, state this explicitly?

Your performance on Quant has no correlation with your 1st Vebal question. I gauged this from GMATPrep - IMO

I have conducted similar experiments too.

My observations:
For Quants getting as many (~20-25) initial question correct helps boost the score. However this does not hold true in case of Verbal. If we were to believe that the GMAT is truly adaptive test, that questions get harder as you answer correctly and that the harder questions carry higher weight, then this whole scheme seems to fall apart for Verbal.

I was wondering if the quickness of your answer affects your score. Does anybody know?

Say if I got lucky and guessed my last 5 questions right in whatever section in the span of 1 minute, would that score be higher if I had taken my time and answered them right in the span of 10 mins?

Thanks,
millhouse

I believe that timing in general does not matter with one exception several alluded to: it may ding you for answering too quickly (within a few seconds) since in that case you are just guessing.

Guys i don't think that the prep software and actual exam algorithms are exactly same..
i've heard, so many people saying that what they observed on the prep was not completely same or true in the actual test..
In actual test, they got very diff. results from the strategies they "assumed" or followed by experimenting with gmatprep..with same difficulty questions..

So, i would rather focus on the concepts..

Superb analysis. It has broken one of my greatest myth. Thanks a lot bsd.

The Manhattan GMAT people have done their own analysis on this concept as well:

https://www.manhattangmat.com/gmat-prep- ... -annex.cfm

I thought I would add a unique expereince with GMAT Prep.

I took the GMATPrep2 yesterday and scored 3 wrong (all in a row) in the first 10. Questions 8, 9 and 10 to be specific. I then scored another 3 wrong on the next 10 questions and 1 wrong in the last 17.

Questions.............# Wrong
1-10.........................3
11-20.......................3
21-37.......................1

Overal Quant score: 50

I really do not think the first 10 or even 20 are as heavily weighted as most people think. MGMAT seems to follow this belief as well.

The wrong answers could be experimental questions, so Q50 is possible.

I missed 9 Quant on the GMATPrep Test. Got first 13 correct and then missed questions evenly for next 23. They were tough. First, time taking test. I got a 50Q. Just thought I would put up my 2 cents. I am a strong believer that first 10 matter most.

Also, missed 9 in V and got a 41. Then, (on different software) missed first 2 and only 4 total and got a 39V!

I got a 730 actually on the GMATprep. I have yet to take the second one and I am saving it for the week before my test (Oct. 10).

I have taken Kaplan twice and got 710 & 760. Manhattan = 730. Princeton = 710. I have traditionally performed well on test day so I am hoping that I get 700+. I think that my app will speak for itself as long as I can get in the game.

Great experiment... Keep going..

I got a 46 in quant with 16 mistakes and 44 in quant with just 11 mistakes.
when i got a 44 i got a series of 3 questions wrong and when i got 46, my mistakes were spread. My guess is that mistakes done continously cost more.

The test must be finished at all costs.
Don't ever get a "bad" streak of answering two or more wrong in a row.