Dear Alex and the rest of the GMAC team,
Happy New Year.
Thank you for your detailed reply; at last GMAC replied.
My postings on this forum are towards the greater good of all stakeholders - the test takers, the test makers, and the B-schools that consider the GMAT score.
The reason of my giving the tennis analogy was to show how “experimental” points mean a world of difference between a fair tennis game that is dependent on tennis skills and a tennis game that is dependent on luck. Similarly the experimental questions turn GMAT (an exam meant to test the aptitude to a test taker) into an exam that tests the luck of a person.
Stating that the number of “experimental” questions is equal for all test does not take away the fact that the “experimental” questions are still there. The test takers do not get a heads-up on the experimental questions, and this is like trapping them into spending time to answer the questions that don’t count.
You put forward the argument that “As with the algorithms, the process for experimental items is the same for every examinee and every instance. Each examinee will receive the same number of experimental items as every other examinee, without regard to current or past performance. The questions that count and the questions that do not are fixed for the administration, and the score would not be manipulated by selecting the experimental items after reviewing performance on all items.
”. A game of dice is quite similar - as with the GMAT, the numbers on the faces are the same for every player and on every instance. Each player will receive the same number of throws as every other player, without regard to current or past performance. The number of faces and the numbers on the faces are fixed, and the score would not be manipulated. Does that makes rolling the dice 37 times similar to attempting 37 quant questions? Similarly, is rolling the dice 41 times similar to attempting 41 verbal questions. And in both instances is it the aptitude that is being tested or is it the luck? And would the B-schools consider a dice player’s score as a reliable indicator of his aptitude – they won’t.
Admittedly, saying that GMAT is exactly similar to a game of dice would be an exaggeration - GMAT still requires aptitude. At the same, I would reiterate that the experimental questions make the GMAT and a game of dice quite similar – dependent on luck. The disappointing part is that GMAC, leave alone doing anything about it, does not admit it.
Another point to note is that of the three main stakeholders in the GMAT, the experimental questions go against two.
1. I have mentioned how they go against the test takers.
2. They also go against the B-schools who trust the GMAT score to be an indicator of a test taker’s aptitude (this is what GMAT mentions as one of the reason of taking the GMAT). The truth is that the GMAT scores are not a true indicator – as pointed earlier, they are influenced by factors similar to factors in a dice game – as you cannot call a dice game score an indicator of pure aptitude, similarly GMAT score cannot be called an indicator of pure aptitude. At the moment, a B-school, assuming the GMAT score to be a perfect aptitude indicator, might screen-in a test taker who scored high and screen-out a test taker who scored less. I am not sure how many B-schools, knowing the true impact of the experimental questions on the scores, will still accept the GMAT scores.
3. As for the third stakeholder – the GMAC, undoubtedly, the GMAT is the easiest place to have the experimental questions – nothing can be easier than throwing in the experimental questions in the real test and letting the gullible test takers fall in trap. While this is easy, the question is – is it ethical? The actual GMAT is not the right place to pounce upon unsuspecting test takers to “test” questions – this is akin to considering the test takers as guinea pigs for GMACs experiments and experimental questions. I am quite sure there are other ways to test questions.
If GMAC still wants to have the experimental questions in the GMAT, then there are better ways to do it – get smarter, tweak the software so that the scores are less dependent on the test taker’s luck and thus showcase the true aptitude of the test. For example, rather than having a set number of experimental questions (for example 10 in a section) – GMAC can tweak the software in a way that kind of makes sure that the number of attempted experimental questions are the same for every test taker. Admittedly, the tweak is going to be complicated but if GMAC wants to have experimental questions as well as a score reflecting the real aptitude then GMAC has to think outside the box. In its present form, the scores do not get the test takers and the B-Schools what they really want – scores uninfluenced by luck.
In the light of the above arguments, I would like to know:
1. How committed is GMAC to have the luck factor taken off the GMAT completely and make the GMAT a test of pure aptitude?
2. What is the need of experimental questions? I am quite sure that the test makers are quite competent - does GMAC doubt their competence?
3. No one exactly knows the number of the experimental questions – is it one, two, three, thirty?
Had luck or experimental questions played no role – there would not have been a drop of 8 points in Quant and an improvement of 8 in Verbal (my aptitude did not change in a month).
Why is it that even after consistently scoring around 690 on prep tests and the GMATPrep tests, I scored 570 on the real GMAT both times? – it has either to do with the faulty software or the “experimental” questions.
Finally, I would again request you to reassess my test manually and show me my tests – at this point in this, this is what transparency calls for.
Sincerely,
Govind Singh Patwal