karna2129 wrote:
Hi Mike,
I have been your keen follower.
The above question is of the 650-700 level.I just wanted to understand in a bit detail about how GMAC actually fixes the level of the question.I mean is it through some sort of Percentile analysis related to that question.
Thanks for responding in advance.
Regards,
KB
Dear
karna2129,
I'm happy to respond.
First of all, please understand: any difficulty estimates given here in GMAT Club are complete guesses, pure estimates. Invest them with absolutely no validity whatsoever. I wrote this particular question, but is it a 650 question? a 700 question? To be perfectly frank, from the question itself, I have no idea. Absolutely no one can look at a question and predict with complete precision the level of difficulty of a question.
Now, for this particular question, I happen to know that since we released it in our product in
Magoosh, almost 2000
Magoosh users have answered it, and approximately 44.7% of them have answered the question correct. Now, are
Magoosh users a good estimate of the entire GMAT taking population? I don't know. I do know that when GMAC tests experimental questions, it gets tens, even hundreds, of thousands of responds: by comparison, fewer than 2000 responses is a relatively small sample.
Let's assume that, of all test takers, 44.7% of them got the question correct. Well, looking at the GMAT percentiles:
https://magoosh.com/gmat/2013/gmat-score-percentiles/If 44.7% get the question right --- that's a percentile that corresponds to about a 560 GMAT score. This is a rough idea. One might call this a "560-question," although that is problematic.
Notice, that even with a great deal of numerical information, the upper reaches of the percentile information become very tricky, because even when people are randomly guessing on five-choice multiple choice, they still have a 25% chance of guessing correctly. If only, say, 3% of the population can actually compute the answer to a question, this small percentage will be hard to separate from all the random guesses who simply, by pure chance, happened to guess correctly. One needs some very sophisticated statistical analyses to make discerning distinctions in that upper part of the Bell Curve.
Keep in mind --- this entire idea of attaching a particular point value to a question is highly suspect, at a number of levels. Notice that GMAC itself doesn't even touch this with a ten-foot pole. GMAC has absolutely no interest in assigning point values to questions, because it's such an inherently dubious business. It's one of these ideas to which student attach considerable importance without fully understanding how logically shifty it is, so we folks who work for private companies and answer student question have to discuss it, and in doing so, we convey the mistaken impression that the entire idea holds any water at all. It's an idea that has gained a great deal of currency, not because it's inherent sound, but only because students mistakenly imagine that such information is critically helpful to them. In any teacher-student relationship, ideas that arise primarily on the student side rather than the teacher side overwhelmingly tend to be considerably less helpful in the long run to the students than they might have imagined. Just because an idea is popular doesn't mean it's good!
For example, suppose one gets this question and questions as hard as this correct, and suppose one gets, say, a 27 on Verbal. Well, as you well know, a 27 on Verbal could be paired with a 15 on Quant or with a 51 on Quant --- those different Quant scores would have virtually nothing to do with one's verbal abilities, but would result in vastly different GMAT scores. If we start comparing difficulty on Verbal vs. Quant questions, we are comparing apples to oranges, because different segments of the population excel at these different tasks. Furthermore, one's GMAT score is not at all a simple thing to understand: the GMAT uses a mind-bogglingly difficult algorithm in the
Computer Adaptive Testing, and those of us without Ph.D.s in Psychometrics could not hope to understand all the nuances, even if we were allowed to see the process they are using. (In fact, that entire algorithm is proprietary knowledge of GMAC, and those of us on the outside know nothing about it!) One's GMAT score in fact depends both on the difficulty of the questions one gets correct and the difficulty of the questions one gets wrong: that's part and parcel of how the CAT calculates your score. If one gets
all the questions of this difficulty correct, vs. half of them right and half of them wrong, those two scenarios would result in very different scores. Performance on one question, in isolation, is virtually meaningless. See:
https://magoosh.com/gmat/2012/what-is-th ... tive-test/I would say: a discerning approach would be to abandon any thought of attaching particular GMAT-scores to particular questions. Through keeping a log of your own mistakes, get familiar with the patterns and with what is hard
for you. Knowing your own weaknesses is considerably more important than knowing overall statistical trends.
Does all this make sense?
Mike