GMAT Prep Software Scoring Analysis and What If Scenarios

NEW ONLINE FORMAT

Couple of years ago we ran different scenarios with GMAT Prep Software to find out whether various myth about scoring algorithm were legit. You can find the results of that analysis in GMAT Prep Software Analysis and What If Scenarios.

On April 16, 2018, the GMAT exam shortened. The number of Quant questions were reduced by 6: from 37 to 31 and the number of Verbal questions will be reduced by 5: from 41 to 36.

Therefore, we are starting testing more or less the same scenarios with the NEW GMAT Prep Software to find out whether anything changed compared to old software.

1. What is the importance of the first 10 questions?

Experiment: We will try to disprove the myth the way OG/GMAC positions it: the first 10 questions are not critical
Methodology: we will attempt the worst case scenario and will answer the first 10 questions incorrectly (not guess but incorrectly); then we will attempt to answer the remaining 21 questions correctly (not guess)
Result: Score - Q30. Percentile - 19th (!!!). First 10 questions WRONG. The next 21 questions CORRECT
Analysis: obviously something is not right with the myth about the first 10 questions. They appear to be very important on the Quant. If you miss the first 10 (probably not a very realistic scenario for most) you have no way of climbing out of the hole - almost all questions that were offered to me were generally easy/medium difficulty.
When simulated in the previous GMAT Prep version, this scenario resulted in Score - Q38. Percentile - 48th. So, the importance of the first 10 questions seems to increase (no surprice since now 10 questions correspond to 1/3 of the questions).
Questions: 18PS and 13 DS.

TO ENSURE THAT THE RESULT ABOVE WAS NOT AN ANOMALY, WE RAN THIS SCENARIO TWO TIMES MORE:
Result: Score - Q31. Percentile - 20th (!!!). First 10 questions WRONG. The next 21 questions CORRECT. Second attempt
Questions: 18PS and 13 DS.
Total Score: every question in the verbal section was answered as "D" this time and it resulted in the final score of V6, 0 percentile. Total score: 320.

Result: Score - Q29. Percentile - 16th (!!!). First 10 questions WRONG. The next 21 questions CORRECT. Third attempt
Questions: 17PS and 14 DS.
Total Score: every question in the verbal section was answered as "C" this time and it resulted in the final score of V9, 2 percentile (Also, interesting becasie C gave a better result than D and E. Though it might be a simple luck). Total score: 320.

2. What is the importance of the last 10 questions?

Experiment: We will try to analyze the importance of the last 10 questions and if it equals the important of the first 10
Methodology: we will attempt the worst case scenario and will answer the first 27 Q questions correctly and then will answer the remaining 10 questions incorrectly (not guess but incorrectly).
Result: Score - Q49. Percentile - 75th. First 21 questions CORRECT. The final 10 questions WRONG.
Analysis: The questions were much harder this time with some new topics/types appearing on the test such as coordinate geometry/probability etc. Conclusion: the last 10 don't count as much as the first 10. This myth is True (as long as you can get the first 21 right). Please note that I have not guessed the last 10 but rather answered them incorrectly (which means I still had to solve those questions too). If you are guessing, you will most likely get a higher score.
When simulated in the previous GMAT Prep version, this scenario resulted in Score - Q50. Percentile - 92th.
Questions: 18PS and 13 DS.

3. What is the importance of the middle 10 questions?

Experiment: We will try to analyze the importance of the middle 10 questions and see how it compares to the results of the previous 2 experiments
Methodology: we will attempt the worst case scenario and will answer the first 11 Q questions correctly; then I will answer incorrectly the following 10 questions, and will answer the remaining 10 questions correctly.
Result: Score - Q49. Percentile - 75th. First 11 questions CORRECT. Next 10 INCORRECT. The final 10 questions CORRECT.
Analysis: The questions were not much easier than in the second scenario above even though I have made 10 consecutive mistakes.
When simulated in the previous GMAT Prep version, this scenario resulted in Score - Q49. Percentile - 85th.
Total Score: every question in the verbal section was answered as "E" this time and it resulted in the final score of V6, 0 percentile. Total score: 440.
Questions: 17PS and 14 DS.

4. What is the penalty if I miss every third question?

Experiment: I will try to test a scenario in which a test takers answers every third question incorrectly (e.g. 3, 6, 9, etc. Thus giving themselves an extra 2 mins on quant). This scenario will simulate someone who takes 3 mins instead of 2 per question and then guesses every third question. Please note that this is the worst case scenario (we do not get any guesses right)
Methodology: Every 3rd questions answered incorrectly. So, 10 incorrect and 21 correct answers.
Result: Score - Q48. Percentile - 69th. Every third questions WRONG.
Analysis: Slightly lower score than in the case of missing the last 10 questions.
When simulated in the old Desktop GMAT Prep version, this scenario resulted in Score - Q49. Percentile - 85th. More details here
Questions: 18PS and 13 DS.

5. Not Answering Last 10 Questions (ran out of time) vs. 10 Wrong Answers

Experiment: We will try to analyze the importance of answering questions in time.
Methodology: we will attempt answering the first 21 questions correctly and then wait till the time run out leaving the last 10 questions unanswered.
Result: Score - Q39. Percentile - 37th vs. Q49. Percentile - 75th. First 21 questions CORRECT. Next 10 NOT ATTEMPTED.
Analysis: I got lower score than in scenario 2 (First 21 questions CORRECT. The final 10 questions WRONG: Score - Q49. Percentile - 75th). This might indicate that answering all questions, finishing a test is very important. So, it's better finish the test and answer all the final questions, even incorrectly than not finish the test and leave final questions unanswered. Only scenario 1 (missing first 10 questions) gave a lower score than this case.
Total Score: every question in the verbal section was answered as "A" this time and it resulted in the final score of V6, 0 percentile. Total score: 370 (8th percentile).
Questions: 12PS and 9DS.

6. Importance of DS questions

Experiment: We will try to analyze the importance DS vs PS.
Methodology: we will attempt answering the first 10 questions correctly and then miss all DS questions, while answering PS correctly.
Result: Score - Q50. Percentile - 86th. First 10 questions CORRECT and then missed 10 DS while answering 11 PS questions correctly.
Total Score: every question in the verbal section was answered as "B" this time and it resulted in the final score of V6, 0 percentile. Total score: 460 (19th percentile).
Questions: 18PS and 13DS.

7. Importance of PS questions

Experiment: We will try to analyze the importance DS vs PS.
Methodology: we will attempt answering the first 10 questions correctly and then miss all PS questions, while answering DS correctly.
Result: Score - Q49. Percentile - 75th. First 10 questions CORRECT and then missed 11 PS while answering 10 DS questions correctly.
Total Score: every question in the verbal section was answered as "D" this time and it resulted in the final score of V8, 1 percentile. Total score: 460 (19th percentile).
Questions: 18PS and 13DS.



_________________

New to the Math Forum?
Please read this: Ultimate GMAT Quantitative Megathread | All You Need for Quant | PLEASE READ AND FOLLOW: 12 Rules for Posting!!!

Resources:
GMAT Math Book | Triangles | Polygons | Coordinate Geometry | Factorials | Circles | Number Theory | Remainders; 8. Overlapping Sets | PDF of Math Book; 10. Remainders | GMAT Prep Software Analysis | SEVEN SAMURAI OF 2012 (BEST DISCUSSIONS) | Tricky questions from previous years.

Collection of Questions:
PS: 1. Tough and Tricky questions; 2. Hard questions; 3. Hard questions part 2; 4. Standard deviation; 5. Tough Problem Solving Questions With Solutions; 6. Probability and Combinations Questions With Solutions; 7 Tough and tricky exponents and roots questions; 8 12 Easy Pieces (or not?); 9 Bakers' Dozen; 10 Algebra set. ,11 Mixed Questions, 12 Fresh Meat

DS: 1. DS tough questions; 2. DS tough questions part 2; 3. DS tough questions part 3; 4. DS Standard deviation; 5. Inequalities; 6. 700+ GMAT Data Sufficiency Questions With Explanations; 7 Tough and tricky exponents and roots questions; 8 The Discreet Charm of the DS; 9 Devil's Dozen!!!; 10 Number Properties set., 11 New DS set.


What are GMAT Club Tests?
Extra-hard Quant Tests with Brilliant Analytics

Awesome Bunuel. I think this is what most of the candidates must be waiting for!
_________________

My journey From 410 to 700
Here's my experience when I faced a glitch in my GMAT Exam
Don't do this mistake when you give your GMATPrep Mock!
NEW GMATPrep software analysis by Bunuel

souvik101990, will you be analyzing other scenarios (as Vercules did) with the new format tests? Such as missing the last 9, questions, missing the middle 9 questions, etc.

New scenario added: 5. Not Answering Last 10 Questions (ran out of time) vs. 10 Wrong Answers
_________________

New to the Math Forum?
Please read this: Ultimate GMAT Quantitative Megathread | All You Need for Quant | PLEASE READ AND FOLLOW: 12 Rules for Posting!!!

Resources:
GMAT Math Book | Triangles | Polygons | Coordinate Geometry | Factorials | Circles | Number Theory | Remainders; 8. Overlapping Sets | PDF of Math Book; 10. Remainders | GMAT Prep Software Analysis | SEVEN SAMURAI OF 2012 (BEST DISCUSSIONS) | Tricky questions from previous years.

Collection of Questions:
PS: 1. Tough and Tricky questions; 2. Hard questions; 3. Hard questions part 2; 4. Standard deviation; 5. Tough Problem Solving Questions With Solutions; 6. Probability and Combinations Questions With Solutions; 7 Tough and tricky exponents and roots questions; 8 12 Easy Pieces (or not?); 9 Bakers' Dozen; 10 Algebra set. ,11 Mixed Questions, 12 Fresh Meat

DS: 1. DS tough questions; 2. DS tough questions part 2; 3. DS tough questions part 3; 4. DS Standard deviation; 5. Inequalities; 6. 700+ GMAT Data Sufficiency Questions With Explanations; 7 Tough and tricky exponents and roots questions; 8 The Discreet Charm of the DS; 9 Devil's Dozen!!!; 10 Number Properties set., 11 New DS set.


What are GMAT Club Tests?
Extra-hard Quant Tests with Brilliant Analytics

This topic is gold. I'll definitely try some of these approaches both on Prep and oficial test.

Hi I am running three new scenarios today.

1. Missing the last 9 questions.
2. Missing the middle 9 questions.
3. Guessing all RCs and marking everything else correctly. Should be up in 12 hours, so stay tuned
_________________

Have an MBA application Question? ASK ME ANYTHING!

My Stuff: Four Years to 760 | MBA Trends for Indian Applicants

My GMAT Resources
V30-V40: How to do it! | GMATPrep SC | GMATPrep CR | GMATPrep RC | Critical Reasoning Megathread | CR: Numbers and Statistics | CR: Weaken | CR: Strengthen | CR: Assumption | SC: Modifier | SC: Meaning | SC: SV Agreement | RC: Primary Purpose | PS/DS: Numbers and Inequalities | PS/DS: Combinatorics and Coordinates

My MBA Resources
Everything about the MBA Application | Over-Represented MBA woes | Fit Vs Rankings | Low GPA: What you can do | Letter of Recommendation: The Guide | Indian B Schools accepting GMAT score | Why MBA?

My Reviews
How I got into five schools from zero - Applicant Lab Review
Veritas Prep Live Online

Also interested in seeing the outcome of scoring high in two sections and low in one (i.e. SC/CR 40+, RC ~30). Thanks for all of your work!

Great stuff, Bunuel! Cheers to all your valuable contributions to the Quant forums.


I ran a question-type count on the new (online) Verbal section of Exam 1, and here is what I found: 15 SC, 13 RC, 8 CR.

Make of that what you will! Also note that this balance/frequency of question Verbal types on the new GMATPrep tests does NOT appear to reflect their true proportions on the actual GMAT, based on the debriefs we've read so far, which suggest that RC is now the most common question type:



My best guess for the actual number of counted questions on the GMAT is this: 15 PS, 13 DS, 12 RC, 10 SC, 8 CR.

Keep in mind that I only ran one simulation. It is entirely possible that every GMATPrep test now has a slightly different balance of question types, much like the real GMAT itself, due to the random distribution of experimental questions by type.

-Brian
_________________

Harvard grad and 99% GMAT scorer, offering expert, private GMAT tutoring and coaching worldwide since 2002.

One of the only known humans to have taken the GMAT 5 times and scored in the 700s every time (700, 710, 730, 750, 770), including verified section scores of Q50 / V47, as well as personal bests of 8/8 IR (2 times), 6/6 AWA (4 times), 50/51Q and 48/51V (1 question wrong).

You can download my official test-taker score report (all scores within the last 5 years) directly from the Pearson Vue website: https://tinyurl.com/y7knw7bt Date of Birth: 09 December 1979.

GMAT Action Plan and Free E-Book - McElroy Tutoring

Contact: mcelroy@post.harvard.edu (I do not respond to PMs on GMAT Club.)

...or find me on Reddit: http://www.reddit.com/r/GMATpreparation

Hi sandman13,

Good information, thanks! This confirms my suspicion that the precise number of questions of each type now varies on each GMAT Prep test, as it does on the real GMAT. It also leads me to believe that 12 RC / 10 SC / 8 CR is indeed the correct split of counted questions.

-Brian
_________________

Harvard grad and 99% GMAT scorer, offering expert, private GMAT tutoring and coaching worldwide since 2002.

One of the only known humans to have taken the GMAT 5 times and scored in the 700s every time (700, 710, 730, 750, 770), including verified section scores of Q50 / V47, as well as personal bests of 8/8 IR (2 times), 6/6 AWA (4 times), 50/51Q and 48/51V (1 question wrong).

You can download my official test-taker score report (all scores within the last 5 years) directly from the Pearson Vue website: https://tinyurl.com/y7knw7bt Date of Birth: 09 December 1979.

GMAT Action Plan and Free E-Book - McElroy Tutoring

Contact: mcelroy@post.harvard.edu (I do not respond to PMs on GMAT Club.)

...or find me on Reddit: http://www.reddit.com/r/GMATpreparation

Hey guys,

Here are my results of 2 mock tests. I don't understand the paradox: in attempt with lower result all 1-20 are correct (theoretically having higher impact). The ratio of correct/incorrect answers is almost the same in both attempts, differing in 1 incorrect answer only. Please see attached files.

Any thoughts?