Last visit was: 22 May 2026, 01:30 It is currently 22 May 2026, 01:30
Close
GMAT Club Daily Prep
Thank you for using the timer - this advanced tool can estimate your performance and suggest more practice questions. We have subscribed you to Daily Prep Questions via email.

Customized
for You

we will pick new questions that match your level based on your Timer History

Track
Your Progress

every week, we’ll send you an estimated GMAT score based on your performance

Practice
Pays

we will pick new questions that match your level based on your Timer History
Not interested in getting valuable practice questions and articles delivered to your email? No problem, unsubscribe here.
Close
Request Expert Reply
Confirm Cancel
avatar
catennacio
Joined: 15 Apr 2010
Last visit: 29 Nov 2022
Posts: 34
Own Kudos:
109
 [1]
Given Kudos: 11
Posts: 34
Kudos: 109
 [1]
1
Kudos
Add Kudos
Bookmarks
Bookmark this Post
User avatar
iVoRy
Joined: 21 Apr 2011
Last visit: 29 Oct 2021
Posts: 140
Own Kudos:
Given Kudos: 36
Concentration: Strategy
GMAT 1: 700 Q48 V38
GPA: 3.09
Products:
GMAT 1: 700 Q48 V38
Posts: 140
Kudos: 31
Kudos
Add Kudos
Bookmarks
Bookmark this Post
User avatar
OfficialGMAT
User avatar
Official GMAC Representative
Joined: 04 Jun 2012
Last visit: 30 Aug 2024
Posts: 447
Own Kudos:
Given Kudos: 7
Expert
Expert reply
Posts: 447
Kudos: 172
Kudos
Add Kudos
Bookmarks
Bookmark this Post
avatar
catennacio
Joined: 15 Apr 2010
Last visit: 29 Nov 2022
Posts: 34
Own Kudos:
Given Kudos: 11
Posts: 34
Kudos: 109
Kudos
Add Kudos
Bookmarks
Bookmark this Post
OfficialGMAT
catennacio
Hey guys,

I took the GMATPrep CAT and found very surprising results which I can't explain:

Yesterday, I took 2 CATs with GMATPrep 2.0 (the very early version of GMATPrep released on April 3rd from mba.com)
I got the results:
- CAT1: 640 (Q48, V30, IR2). Quant = 24/37 right (13 wrong). Verbal = 26/41 right (15 wrong). I didn't know that I couldn't go back the review screen so I ended the test.

Fine, I decided to take it again on the afternoon because in CAT1 somebody called me on the phone and we talked for 5 mins:
- CAT2: 700 (Q49, V35, IR2)
Quant = 26/37 (11 wrong)
Verbal = 27/41 (14 wrong).
This time, I reviewed my session and found that the software has flaws in detecting right answers in IR (I heard somewhere on this forum).

So today I went to mba.com and downloaded the 2.1.277 version (latest one that they have) and retook the test again... and here comes the shocking results:
- CAT1: 590 (Q49, V21, IR7)!!!
Quant = 28/37 (9 wrong).
Verbal = 27/41 (14 wrong)

Quant is fine, although I still slightly doubt that to have a Q49 on the 2.1.277 version is more difficult than to have a Q49 on the 2.0 version. This is illustrated by the fact that my Quant hit rate has to be 28/37, as compared to 26/37 in the previous version, in order to have a Q49. OK, easy questions and difficult questions factors come into play, so I'm OK with that.

However I can't explain how my Verbal fluctuates that much, provided that I have the same hit rate in the new version and the old version: both at 27/41 (14 wrong). In the previous version, I got a V35 whereas in the new version I only got a V21!!

My questions:
- If my hit rate is around 27/41, what is the APPROXIMATE V score do I get after all? It can't fluctuate from V21 to V35 as in my case, no matter how they tweaked the scoring algorithm.
- Does the scoring algorithm differentiate among wrong answers? I mean does it consider an answer to be wrong, very wrong, very very wrong and extremely wrong? This is the only explanation I can offer to myself to understand how the algorithm works. So even if I got the same number of wrong answers, because I chose a lot of extremely wrong answers, the software thinks I'm so stupid that it gives me a V21 instead of a V35.
- Has anyone experienced this before? How did you explain it?

Thanks!

Hello, Catennacio! Rebecca from GMAC here. I saw your query and showed it our GMATPrep team here internally. Here is their response:

Total score is based on raw performance on both the Verbal and Quantitative sections. A range of raw scores will map onto a single scaled score, but since the Total score is based on the raw performance, greater differentiation is possible, meaning the same Quant and Verbal scores can map to different Total scores. It is much like rounding error. Consider scaled scores of 40 for Quant and 35 for Verbal. Say, on the first test, the ‘unrounded’ scores were actually 39.51 and 34.51, and on the second test the ‘unrounded’ scores were 40.49 and 35.49. The difference is only apparent when the scores are combined (sum of 74.02 compared to 75.98). In practice, Total scores up to about 30-40 points apart are considered very similar and should be interpreted as nearly equivalent.

As you know, the CAT format means your score is calculated based on not just how many questions you answered correctly but also the difficulty level. That could also account for a discrepancy in your score. We have not seen similar accounts of large score discrepancies based on the different versions of GMATPrep, but if others have had the same experience please do share them here so we can look into it further. Catennacio, please feel free to call our customer service line directly at 1 (800) 717-GMAT (4628) if you want to discuss it further.

Thank you!

Hi Rebecca,

Thanks for your response. I PM'ed you. Please check.

Edit:

Serious stuffs first, my question:

- I plan to take the GMAT soon, so I want to know whether the software used at the test centers is identical in terms of design, scoring mechanism and such? I understand there are differences between the GMATPrep and the real GMAT Test Console such as there is no timed break on the GMATPrep and you can review your score after all, etc.. Those are OK. What I concern is that if the design (say, software design) and scoring algorithm are the same, and the latest GMATPrep seems to have bugs, I don't want to risk taking the real GMAT with that same "flawed" algorithm. Also, how do you update the test centers with the latest software that hopefully has the least bugs? This is very crucial to know if one plans to take the real GMAT soon.

More thoughts on the software: (please excuse me if my imagination and the software engineering part of mine bite in the wrong place)

- By judging from the fact that one can resume his test session in case of any interruption, such as loss of power or say, any error on the operating system, it is deducible that GMAT software (both real software and prep software) use their log files to calculate the score after the test. I experienced this because my last real GMAT test was interrupted due to power failure and I was able to resume to where I left. I posted the issue here: do-i-have-a-valid-case-to-complain-about-the-test-center-132369.html

Therefore, if the GMATPrep 2.1.277 uses the such way to calculate scores (by looking at the log files in the end to retrieve the right/wrong answers), the paradox that my V score was too low while my hit rate was high, can be thought as the following:

- When the test taker chooses an answer and click Next, GMATPrep saves the choice into its memory AS WELL AS its log file. When the exam is finished, GMATPrep uses the log file to retrieve the answer choices made by the test taker during the whole test to calculate the final band score, and hence the scale score. This is the right approach since GMATPrep doesn't want to read the answer choices from its memory because if there is a power cut-off, all the information in the computer memory is wiped. If the Logging component has bugs (for instance, hard drive is full, or hard drive is error, or Adobe Air doesn't have the privilege to write onto the host system etc), the log file might not write the correct info that is needed later on to calculate the score.

- The Review screen (see attached image) shows the number of right and wrong questions, and this information is taken directly from the memory. So if the operation to write the answer choices into the memory is correct and the Logging component is not stable, such paradox happens as explained in my first post: V hit rate high (from memory) but the calculated V score is low (from log file because of bugs).

- If the above scenario is the case, having your team look at the Logging component will help them detect the error. By the way, why does GMAC use Adobe Air to deliver such an important piece of software? Air has never been a "mainstream" framework on which important softwares are developed...

Hope this helps!

-C
Attachments

strangeCAT.jpg
strangeCAT.jpg [ 116.88 KiB | Viewed 3347 times ]

User avatar
OfficialGMAT
User avatar
Official GMAC Representative
Joined: 04 Jun 2012
Last visit: 30 Aug 2024
Posts: 447
Own Kudos:
Given Kudos: 7
Expert
Expert reply
Posts: 447
Kudos: 172
Kudos
Add Kudos
Bookmarks
Bookmark this Post
catennacio
OfficialGMAT
catennacio
Hey guys,

I took the GMATPrep CAT and found very surprising results which I can't explain:

Yesterday, I took 2 CATs with GMATPrep 2.0 (the very early version of GMATPrep released on April 3rd from mba.com)
I got the results:
- CAT1: 640 (Q48, V30, IR2). Quant = 24/37 right (13 wrong). Verbal = 26/41 right (15 wrong). I didn't know that I couldn't go back the review screen so I ended the test.

Fine, I decided to take it again on the afternoon because in CAT1 somebody called me on the phone and we talked for 5 mins:
- CAT2: 700 (Q49, V35, IR2)
Quant = 26/37 (11 wrong)
Verbal = 27/41 (14 wrong).
This time, I reviewed my session and found that the software has flaws in detecting right answers in IR (I heard somewhere on this forum).

So today I went to mba.com and downloaded the 2.1.277 version (latest one that they have) and retook the test again... and here comes the shocking results:
- CAT1: 590 (Q49, V21, IR7)!!!
Quant = 28/37 (9 wrong).
Verbal = 27/41 (14 wrong)

Quant is fine, although I still slightly doubt that to have a Q49 on the 2.1.277 version is more difficult than to have a Q49 on the 2.0 version. This is illustrated by the fact that my Quant hit rate has to be 28/37, as compared to 26/37 in the previous version, in order to have a Q49. OK, easy questions and difficult questions factors come into play, so I'm OK with that.

However I can't explain how my Verbal fluctuates that much, provided that I have the same hit rate in the new version and the old version: both at 27/41 (14 wrong). In the previous version, I got a V35 whereas in the new version I only got a V21!!

My questions:
- If my hit rate is around 27/41, what is the APPROXIMATE V score do I get after all? It can't fluctuate from V21 to V35 as in my case, no matter how they tweaked the scoring algorithm.
- Does the scoring algorithm differentiate among wrong answers? I mean does it consider an answer to be wrong, very wrong, very very wrong and extremely wrong? This is the only explanation I can offer to myself to understand how the algorithm works. So even if I got the same number of wrong answers, because I chose a lot of extremely wrong answers, the software thinks I'm so stupid that it gives me a V21 instead of a V35.
- Has anyone experienced this before? How did you explain it?

Thanks!

Hello, Catennacio! Rebecca from GMAC here. I saw your query and showed it our GMATPrep team here internally. Here is their response:

Total score is based on raw performance on both the Verbal and Quantitative sections. A range of raw scores will map onto a single scaled score, but since the Total score is based on the raw performance, greater differentiation is possible, meaning the same Quant and Verbal scores can map to different Total scores. It is much like rounding error. Consider scaled scores of 40 for Quant and 35 for Verbal. Say, on the first test, the ‘unrounded’ scores were actually 39.51 and 34.51, and on the second test the ‘unrounded’ scores were 40.49 and 35.49. The difference is only apparent when the scores are combined (sum of 74.02 compared to 75.98). In practice, Total scores up to about 30-40 points apart are considered very similar and should be interpreted as nearly equivalent.

As you know, the CAT format means your score is calculated based on not just how many questions you answered correctly but also the difficulty level. That could also account for a discrepancy in your score. We have not seen similar accounts of large score discrepancies based on the different versions of GMATPrep, but if others have had the same experience please do share them here so we can look into it further. Catennacio, please feel free to call our customer service line directly at 1 (800) 717-GMAT (4628) if you want to discuss it further.

Thank you!

Hi Rebecca,

Thanks for your response. I PM'ed you. Please check.

Edit:

Serious stuffs first, my question:

- I plan to take the GMAT soon, so I want to know whether the software used at the test centers is identical in terms of design, scoring mechanism and such? I understand there are differences between the GMATPrep and the real GMAT Test Console such as there is no timed break on the GMATPrep and you can review your score after all, etc.. Those are OK. What I concern is that if the design (say, software design) and scoring algorithm are the same, and the latest GMATPrep seems to have bugs, I don't want to risk taking the real GMAT with that same "flawed" algorithm. Also, how do you update the test centers with the latest software that hopefully has the least bugs? This is very crucial to know if one plans to take the real GMAT soon.

More thoughts on the software: (please excuse me if my imagination and the software engineering part of mine bite in the wrong place)

- By judging from the fact that one can resume his test session in case of any interruption, such as loss of power or say, any error on the operating system, it is deducible that GMAT software (both real software and prep software) use their log files to calculate the score after the test. I experienced this because my last real GMAT test was interrupted due to power failure and I was able to resume to where I left. I posted the issue here: do-i-have-a-valid-case-to-complain-about-the-test-center-132369.html

Therefore, if the GMATPrep 2.1.277 uses the such way to calculate scores (by looking at the log files in the end to retrieve the right/wrong answers), the paradox that my V score was too low while my hit rate was high, can be thought as the following:

- When the test taker chooses an answer and click Next, GMATPrep saves the choice into its memory AS WELL AS its log file. When the exam is finished, GMATPrep uses the log file to retrieve the answer choices made by the test taker during the whole test to calculate the final band score, and hence the scale score. This is the right approach since GMATPrep doesn't want to read the answer choices from its memory because if there is a power cut-off, all the information in the computer memory is wiped. If the Logging component has bugs (for instance, hard drive is full, or hard drive is error, or Adobe Air doesn't have the privilege to write onto the host system etc), the log file might not write the correct info that is needed later on to calculate the score.

- The Review screen (see attached image) shows the number of right and wrong questions, and this information is taken directly from the memory. So if the operation to write the answer choices into the memory is correct and the Logging component is not stable, such paradox happens as explained in my first post: V hit rate high (from memory) but the calculated V score is low (from log file because of bugs).

- If the above scenario is the case, having your team look at the Logging component will help them detect the error. By the way, why does GMAC use Adobe Air to deliver such an important piece of software? Air has never been a "mainstream" framework on which important softwares are developed...

Hope this helps!

-C

Hello, C!

I had our product team review the log files you sent us. They looked through your scores, and did not find any errors with the scoring.

Additionally, I showed them your most recent post. They would like me to share the following information with you:

Concerns about the actual GMAT® algorithm
There were two issues with the GMATPrep software which have since been resolved:
» IR scoring
» Quant and Verbal scoring when a candidate stopped an exam and went back to it later (something not possible during the actual exam)
Both have been resolved and relate to the GMATPrep® software vs the question selection and scoring algorithm

Theories on how the software works
Log files are created for most software products in order to help with debugging and issue analysis. The GMATPrep® software is the same. Based on our analysis of your log files, your Quant and Verbal scores from all 3 exams are correct.


Please let me know if this addresses your concerns!

Thank you!

Rebecca
Moderators:
203 posts
General GMAT Forum Moderator
474 posts