Edit: Posting here for the record
Official GMAT Score vs. GMAT Prep AnalysisTwo weeks ago,
we started collecting GMAT Scoring information for the calculator. Thank you everyone who submitted your GMAT Scores - please continue to submit your scores! The more data we have the better.
At this time, this is a fairly small sample of only 72 Real GMAT Attempts and corresponding GMAT Prep Scores. Please do not use this data for any conspiracy theories YET! Here is where we are today Aug 18, 2021:Submissions with Practice and Official Scores: GMAT-Online Scores:
31GMAT Test Center Scores:
41Lacking Official GMAT Scores:
187Total Data Submission Attempts:
259I wanted to get closer to 100 data points but I feel having even 70+ is sufficient to start the analysis. Also, the positive side of the story is that many of the test-takers submitted multiple GMAT Prep Scores and third party practice test scores. It is probably too early to be able to measure the third party tests yet (need more data - so please submit your scores if you have taken the GMAT) but I have been able to pull some GMAT Prep Analysis. I have to give a disclaimer that I am not a statistician or pretending to be one and I welcome any suggestions how to handle the data. My analysis is my best attempt at making these findings useful. If you are a data expert and have suggestions, please let me know.
What is the correlation between GMAT Prep and GMAT Online scores?
Correlation and Standard Deviation GMAT Prep vs. Official ONLINE GMAT |
| Exam 1 | Exam 2 | Exam 3 | Exam 4 | Exam 5 | Exam 6 |
Correlation | 0.68 | 0.68 | 0.59 | 0.81 | 0.54 | 0.51 |
Mean Difference | 9 | 14 | 8 | 32 | 39 | 13 |
STDEV | 38 | 38 | 42 | 27 | 25 | 18 |
Sample Size: | 24 | 21 | 13 | 12 | 10 | 8 |
The samples represent 80% of data; 20% of scores that were the biggest outliers were removed.
The Mean difference is the average of Official GMAT minus GMAT Prep Exam Score
The STDEV is standard deviation of the difference between a particular Exam and the Official Test. STDEV gives you an idea how 67% of people have scoredONLINE GMAT Bottom Line: Looking at the Correlation, it is interesting how consistent Exams 1 and 2 are.
Exam 4 is the most correlated with the Official GMAT and I think that is likely because people took it right before the test so I feel the correlation is circumstantial rather than one test being more accurate than the other. It all depends on what was the last GMAT Prep one took and how close to the test. In general correlation of 0.5 means MODERATE and 0.7 means Strong.
I tried discarding "UNFAIR" GMAT Prep tests (these are the ones that people run into questions that they know or perhaps are sick while taking them or are interrupted) from the list of 24 Exams 1 reducing it down to 15 records. Surprisingly it had no material effect on the Correlation or the STDEV parameters that changed by less than 5% and correlation actually went down to 0.66 which was a surprising finding.
The shocking part was that out of 31 Online Tests, more than 50% had issues, thus potentially muddying up water- 13 tests were OK/Normal
- 13 Tests had Lag issues
- 3 Had noise/interference issues
- 2 tests had "other" issues
What is the correlation between GMAT Prep and GMAT Test Center scores?
Correlation and Standard Deviation GMAT Prep vs. Official TEST CENTER GMAT |
| Exam 1 | Exam 2 | Exam 3 | Exam 4 | Exam 5 | Exam 6 |
Correlation | 0.9 | 0.76 | 0.79 | 0.64 | 0.45 | 0.88 |
Mean Difference | 4 | 0 | 11 | 21 | 9 | 33 |
STDEV | 29 | 34 | 30 | 41 | 45 | 22 |
Sample Size: | 21 | 12 | 14 | 17 | 9 | 7 |
The samples represent 80% of data; 20% of scores that were the biggest outliers were removed.
The Mean difference is the average of Official GMAT minus GMAT Prep Exam Score
The STDEV is standard deviation of the difference between a particular Exam and the Official Test. STDEV gives you an idea how 67% of people have scored
I eliminated all of the scenarios when users did not have a realistic score due to repeat questionsTest Center GMAT Bottom Line: I had a bigger sample of the test-center based scores so I eliminated any situation when the score was "NOT-OK" due to repeat questions. I also did not want to focus too much on Test Center vs. Online quite yet as I feel I don't have enough samples .
There are some surprises however, I am getting a very high correlation for the GMAT Prep 1, surprisingly.
I have seen a number of people score in the magnitude of 100 points above/below sometimes but those are outliers and I generally have discarded them in the 20% of the results that are trimmed (not sure if that's right or not)
Overall Conclusion:The data sample is still pretty small being only about 70-something real GMAT scores (and if we trim 20% of the outliers, then that's just 55).
However, there are some interesting observations in both Online and Test-Center GMAT formats: STDEV (which indicates how likely you are to be within the target score from GMAT Prep) is around 30-40 with a few small-sample exams being 20. That sort of indicates the old believe that your score will be within 40 points.
There is some variance and additional research/variations are needed when we have more scores (please submit
) so that we can take only the LAST GMAT Prep score.
It seems there is a lot more variance happening with the ONLINE Test version (perhaps due to over 50% experiencing some kind of an issue)
Welcome thoughts and Ideas!