Weird Verbal Scores on actual GMAT

Sorry for you guys.

There is so many people mention this issue that I have just written to gmac to bring it to their attention.
Let's see what they say...

I'm taking the test on Monday. I haven't scored less than Q 47 / V 40 - 710 (my first attempt) on Gmatprep or Powerprep. I'll post my score on Monday or Tuesday.

For a),
The GMAT scoring algorithm is based on 40 years of research, so there's no way they'll make any significant changes. Still, they have scope to change certain parameters (for example, they can decide, more or less freely without affecting the integrity of the scoring, how much more difficult the second question is than the first if you answer the first correctly), so it's possible they've made minor adjustments. If scores are off, as anecdotal evidence on this and other forums suggests (though my students have consistently scored in the 40s on the verbal on recent GMATs, so I don't know what to think), it may not be the algorithm at fault. The ScoreTop issue may have affected question calibration, which could affect current scores. It's not entirely straightforward to explain clearly, but I'll try:

-suppose a diagnostic question is inserted on a test, and ends up in a ScoreTop JJ document, so every test taker who reads the JJs knows the correct answer in advance. Suppose also that there are *a lot* of people reading the JJs;

-the question appears as a diagnostic. Imagine that this question is, in truth, ballistically difficult- a true 51-level verbal question. GMAC doesn't know how difficult the question is until test-takers see it, as a diagnostic. If a lot of ScoreTop readers know the answer in advance, and get it right, GMAC will think, by analyzing responses, that the question is in fact quite easy.

-the question then shows up on a real GMAT, calibrated as an 'easy' question. Most honest test-takers will see it and get it wrong (after all, in truth it is bleeding hard). But because the test thinks the question is easy, test-takers will get heavily penalized for answering incorrectly, dragging down their scores (especially if this happens on more than one question). If the question had been correctly calibrated as a 'damn difficult' question, there would be almost no penalty for getting it wrong; indeed it would be expected.

That example is exaggerated- the effect wouldn't be so pronounced- but if many questions were affected, scores could be affected in an exaggerated way. So, if we grant that there is a problem with recent verbal scores (and while I don't doubt many think that's the case, I would need more evidence to be convinced), it may have nothing to do with the algorithm, and nothing to do with computer error, and may be down to calibration error. There are many places to look if there is a scoring problem. That said, based on the numbers available in the GMAC v ScoreTop court documents, I did a very quick, and very rough, estimate of how much calibration might be affected (posted in the ScoreTop thread on this forum), and my conclusion was that no one should have suffered more than a one point scaled score loss because of this calibration effect. If the number of ScoreTop users was substantially higher than what I found in the court documents, the effect could be much greater.

To answer b), there's an underlying assumption that the Quant section of the GMAT is testing a single ability, and that the verbal section is testing a (different) single ability. It's a very questionable assumption; mathematical ability is, to my mind, a combination of many separate abilities. The verbal section seem to comprise even more disparate abilities- SC, CR and RC seem to test very different things- so one would expect the verbal scores to have greater 'standard error' than quant scores- you would expect your score on the verbal to depend on how many of each question type you see, and where your abilities lie. Still, the GMAT is designed to be have enough questions that standard error is minimized, and I've yet to see any research that indicates that the verbal section has greater standard error than the math; while I intuitively think the verbal section should have less reliable scores than the quant, that's really just speculation.

Thank you very much for such a detailed explanation, Ian Stewart! Kudos!

I thought that GMAC has taken away Scoretop’s live questions, according to this info from FAQ at https://www.mba.com:



However, there are other sites with ‘bad’ content, so I see why the scores could be affected, especially if those sites have a large number of participants. At the same time, IMHO, it is rather doubtful that this fact is the main cause, firstly, because your estimation shows that the possible amount of the drop is probably insignificant, and, secondly, because all those sites have been there for some time already, and this issue the people discuss is quite recent.

Also, I agree with you that the trend with verbal scores is not evident. I read posts recently where people said that their verbal in fact went up from what they had expected. And I’m sure that there are many of GMAT takers whose scores were just in the expected range. That’s why I thought not of a mistake in the test, and not of the change in the algorithm that makes the scores consistently lower, but of some subtle alteration that has made verbal scores less reliable. But certainly, you are right, and it would be prematurely to state anything of the kind...

Thanks again!

Nice reasoning! However, we don't know whether the difficulty level of questions, once determined, is not verified by the thousands of people who do the exam. Also, people who write questions for GMAC have a fair idea of the difficulty level
of their questions.
Keep in mind that in CATs, no one question is terribly important. Only if people knew the answers to several questions would their scores be distorted.

I have students doing the exam every week, and I do not notice any changes in scoring

I do feel that gmatprep verbal is now not an accurate predictor of verbal performance, whether due to scoring, difficulty, or due to a lot of repeats from the OG. Not to mention that if you see 2 or 3 questions on gmatprep V, that could be a huge swing in verbal where in quant wrong answers seem more forgiving.

My take on V: It is harder than GmatPrep. In my opinion it probably gives you an accurate score as to your level. Like I've stated before, I got the same verbal in practice a year before I took the actual test. I took the test twice a year later and received the same verbal scores all 3 times. I put in hundreds of hours studying , learning all the SC rules, becoming an expert at CR questions and RC, with a 90% hit rate. Yet on the actual exam, it saw through all that BS and got down to my actual verbal skill level, which probably in reality is about right.

Then again Walker got a 40 on V and he learned english like a year ago, so that makes me feel a little incompetent ! ha but he is a genius so I'll let that slide.

So far my email to Pearson has not received any response, from their side .
Their website says that only AWA can be revaluated for which they charge 45$ , and a seperate form needs to be filled.So Im unsure on how my mail will be recieved.

Regarding GMATPREP not being the correct indicator of GMAT score, I agree on the same, but do beleive that if other tests like Princeton, Kaplan, ARCO, GMATCLUB , 800 score are taken there is a strong possibility that GMAT Score will fall in the range taken across these tests.

Unfortunately the core issue here, is that the actual Verbal scores of 25/27/28 are much lower than the range that these tests have shown. The Quant score for all students is still very much in range.

I agree with you terp26. GMATPrep is no longer the right representative for verbal section(once you are through with OG10). I have realized that making 2-3 CR questions that are from OG10 correct can make huge differences on test day. One can save time upto one and half seconds and make that correct if the GMATPrep question is from OG10. Therefore on real test all new questions from CR look difficult in one shot because time is 75mins only.

Kevin and Greenoak - those are great points, and I agree with them. Now that the ScoreTop questions are likely retired, this calibration effect could only come from other illegal forums, but there might have been an effect on scores over the past couple of years, before the ST questions were removed from the pool. Just wanted to respond to a couple of points:



Judging by the amount of research they do, I'm sure GMAC does a lot of post-test analysis. I'm not sure, however, that they assess test-taker performance on questions and compare that with the calibrated question difficulty (or perhaps they almost always discover no discrepancy there), but my only basis for saying that is that it's uncommon for an official score report to be different from the score report you see on the screen after you finish your test.



This is my experience as well.

Count me in too. I have consistently scored higher on the verbal, no matter it is Kaplan, Manhattan, PR, or GMATprep. My lowest verbal on those practice test was 38. And the actual test I scored 27 on verbal. I don't understand. It must be the new trend.

It's interesting to know that you guys felt the same. I also never scored less than 36 in GMATprep, but I eneded with a 28 in verbal on July 3rd.

I'd love to believe the GMAT has changed recently, but probably not. That kind of change would be published someplace. I suffered the same fate on the GMAT. I scored a v39 my first take, and a V25 on my second. More than likely I lost focus rather than actually experiencing a test change. Verbal is all about concentration and focus. I truly believe I was not on my game that day, and I believe all these low verbal scores are simply people running out of steam on verbal when they need it the most.

That can be the most logical assesment of this situation... But I think it is a little bit more than that... There may have been a slight tweak in the Verbal scoring algorithm.... As there seem to be fewer mid range verbal scores in the 30's now... Its either mid 20's or in the 40's.There was another 25 again today scored in Verbal by Barfer.

Maybe I'm looking at a small pool of test takers on GMATCLUB , to make that assesment.

I think the only solution here in Verbal is "walkers strategy to fight for every question " . Unless someone can figure out the concrete reason for these scores... as currently no test prep software can currently give an honest assesment of the Verbal test score...

one other thing to remember is that GmatPrep is only a good indicator in the early game. Many people will have skewed scores because of repeats for the following reasons:

-they take the GmatPrep more than once.
-they thoroughly go through OG11 before GmatPrep (there is overlap)
-they attempt the GmatPrep material posted online before Gprep.

In all 3 scenarios your score will be inflated by GmatPrep.

Took the gmat today.
710 - 48Q - 39V. Pretty much in line with PowerPrep/GmatPrep actually. There were questions in PP/GP that I already knew from the OG's.

Each time I did the exam, my attention waned a bit on the verbal section, as I wasn't used to doing AWA and GMAT in one sitting. If it happened to me, it may happen to others.

lol...and it really affected your score. Seriously though, if (51Q,50V) is 790, what does a person have to do to get an 800?

Here's the answer from GMAC: