OfficialGMAT
catennacio
Hey guys,
I took the GMATPrep CAT and found very surprising results which I can't explain:
Yesterday, I took 2 CATs with GMATPrep 2.0 (the very early version of GMATPrep released on April 3rd from mba.com)
I got the results:
- CAT1: 640 (Q48, V30, IR2). Quant = 24/37 right (13 wrong). Verbal = 26/41 right (15 wrong). I didn't know that I couldn't go back the review screen so I ended the test.
Fine, I decided to take it again on the afternoon because in CAT1 somebody called me on the phone and we talked for 5 mins:
- CAT2: 700 (Q49, V35, IR2)
Quant = 26/37 (11 wrong)
Verbal = 27/41 (14 wrong).
This time, I reviewed my session and found that the software has flaws in detecting right answers in IR (I heard somewhere on this forum).
So today I went to mba.com and downloaded the 2.1.277 version (latest one that they have) and retook the test again... and here comes the shocking results:
- CAT1: 590 (Q49, V21, IR7)!!!
Quant = 28/37 (9 wrong).
Verbal = 27/41 (14 wrong)
Quant is fine, although I still slightly doubt that to have a Q49 on the 2.1.277 version is more difficult than to have a Q49 on the 2.0 version. This is illustrated by the fact that my Quant hit rate has to be 28/37, as compared to 26/37 in the previous version, in order to have a Q49. OK, easy questions and difficult questions factors come into play, so I'm OK with that.
However I can't explain how my Verbal fluctuates that much, provided that I have the same hit rate in the new version and the old version: both at 27/41 (14 wrong). In the previous version, I got a V35 whereas in the new version I only got a V21!!
My questions:
- If my hit rate is around 27/41, what is the APPROXIMATE V score do I get after all? It can't fluctuate from V21 to V35 as in my case, no matter how they tweaked the scoring algorithm.
- Does the scoring algorithm differentiate among wrong answers? I mean does it consider an answer to be wrong, very wrong, very very wrong and extremely wrong? This is the only explanation I can offer to myself to understand how the algorithm works. So even if I got the same number of wrong answers, because I chose a lot of extremely wrong answers, the software thinks I'm so stupid that it gives me a V21 instead of a V35.
- Has anyone experienced this before? How did you explain it?
Thanks!
Hello, Catennacio! Rebecca from GMAC here. I saw your query and showed it our GMATPrep team here internally. Here is their response:
Total score is based on raw performance on both the Verbal and Quantitative sections. A range of raw scores will map onto a single scaled score, but since the Total score is based on the raw performance, greater differentiation is possible, meaning the same Quant and Verbal scores can map to different Total scores. It is much like rounding error. Consider scaled scores of 40 for Quant and 35 for Verbal. Say, on the first test, the ‘unrounded’ scores were actually 39.51 and 34.51, and on the second test the ‘unrounded’ scores were 40.49 and 35.49. The difference is only apparent when the scores are combined (sum of 74.02 compared to 75.98). In practice, Total scores up to about 30-40 points apart are considered very similar and should be interpreted as nearly equivalent.
As you know, the CAT format means your score is calculated based on not just how many questions you answered correctly but also the difficulty level. That could also account for a discrepancy in your score. We have not seen similar accounts of large score discrepancies based on the different versions of GMATPrep, but if others have had the same experience please do share them here so we can look into it further. Catennacio, please feel free to call our customer service line directly at 1 (800) 717-GMAT (4628) if you want to discuss it further.
Thank you!
Hi Rebecca,
Thanks for your response. I PM'ed you. Please check.
Edit:
Serious stuffs first, my question:
- I plan to take the GMAT soon, so I want to know whether the software used at the test centers is identical in terms of design, scoring mechanism and such? I understand there are differences between the GMATPrep and the real GMAT Test Console such as there is no timed break on the GMATPrep and you can review your score after all, etc.. Those are OK. What I concern is that if the design (say, software design) and scoring algorithm are the same, and the latest GMATPrep seems to have bugs, I don't want to risk taking the real GMAT with that same "flawed" algorithm. Also, how do you update the test centers with the latest software that hopefully has the least bugs? This is very crucial to know if one plans to take the real GMAT soon.
More thoughts on the software: (please excuse me if my imagination and the software engineering part of mine bite in the wrong place)
- By judging from the fact that one can resume his test session in case of any interruption, such as loss of power or say, any error on the operating system, it is deducible that GMAT software (both real software and prep software) use their log files to calculate the score after the test. I experienced this because my last real GMAT test was interrupted due to power failure and I was able to resume to where I left. I posted the issue here:
do-i-have-a-valid-case-to-complain-about-the-test-center-132369.htmlTherefore, if the GMATPrep 2.1.277 uses the such way to calculate scores (by looking at the log files in the end to retrieve the right/wrong answers), the paradox that my V score was too low while my hit rate was high, can be thought as the following:
- When the test taker chooses an answer and click Next, GMATPrep saves the choice into its memory AS WELL AS its log file. When the exam is finished, GMATPrep uses the log file to retrieve the answer choices made by the test taker during the whole test to calculate the final band score, and hence the scale score. This is the right approach since GMATPrep doesn't want to read the answer choices from its memory because if there is a power cut-off, all the information in the computer memory is wiped. If the Logging component has bugs (for instance, hard drive is full, or hard drive is error, or Adobe Air doesn't have the privilege to write onto the host system etc), the log file might not write the correct info that is needed later on to calculate the score.
- The Review screen (see attached image) shows the number of right and wrong questions, and this information is taken directly from the memory. So if the operation to write the answer choices into the memory is correct and the Logging component is not stable, such paradox happens as explained in my first post: V hit rate high (from memory) but the calculated V score is low (from log file because of bugs).
- If the above scenario is the case, having your team look at the Logging component will help them detect the error. By the way, why does GMAC use Adobe Air to deliver such an important piece of software? Air has never been a "mainstream" framework on which important softwares are developed...
Hope this helps!
-C