Q51 V50 790! (Missed 1 DS question)

I just verified my score. Thank you for alerting me to that!

When I reached out to GMAC I only asked the 3rd question. The 1st and 2nd questions definitely merit an answer, so I will ask GMAC those questions if they do not address them in their followup.

As you mentioned, the 3rd question is the only one they really "have to" address, but I hope they will divulge more information.

For the record, I'm not trying to criticize the OP at all...I would have done exactly the same thing! (And yes, I have asked for an A+ instead of an A before, too.) As I wrote earlier, I think he has a very valid argument to make, and that he deserves a perfect 800.

Happy to help and congrats on the verified 790! I wish you luck in disputing your score up to an 800 or a 51/51/790, or at least getting a better explanation from GMAC.

There are several misapprehensions in this thread, and a lot of incorrect speculation, so I'll clarify a few things:

- I've never seen a split other than Q51/V51 produce an 800 score (though I could imagine a Q51/V50 might produce an 800), so the tables the OP was referring to, which claim a Q51/V49 produces an 800, are almost certainly incorrect. You can just google "Q51 V49" to see what score that split typically produces (a 790). I've even seen a trustworthy Q51/V51 produce a 790 score (link here). There's a good reason for this - not all Q51 scores are equal. You can get a Q51 with some wrong answers, and one test taker's Q51 might really be closer to a 50.5, while another's might be a 'true' 51. It's the raw scores, before they're rounded off to the 6-51 scale, that are used to produce the score out of 800, and to get an 800 overall score, I'm fairly sure you'd need a 'perfect' Q51;

- because that's the case, it may very well be true in the OP's situation that it's the mistake in Quant that is producing the 790, rather than some issue with the Verbal score. I can't be sure about that one way or the other though;

- your performance on experimental questions has absolutely no influence on your Quant, Verbal or Total scores. If a question is experimental, that means GMAC doesn't even know yet the difficulty level of the question. They don't even know if the question is 'legitimate' (meaning, they don't know if the question has the right statistical properties to even be usable in an IRT-based test). They couldn't even contemplate using responses to experimental question to determine your score, because there'd be no way to know how to even use them, or if they even mean anything at all;

- your performance in the Quant section has absolutely no influence on your Verbal score, and vice versa, for obvious reasons;

- your Verbal score does not indicate that there is a problem with the algorithm. Instead, it indicates there's a problem with the question bank. The OP had the right answer all along. The problem was simply that the test didn't provide enough hard questions to make a distinction between a V50 and a V51. You can see that in the ESR - there weren't a lot of hard Verbal questions on the test. If you know how probabilistic scoring works, it should be clear why that situation might produce a V50 score, even with 100% accuracy.

To make a somewhat extreme analogy: imagine you were obligated to bet a lot of money on how well somebody you don't know will do on the GMAT in Quant. You watch her answer a few practice questions, questions like -- "what is 3+2?", "what is the area of a circle of radius 5?", "what is the median of the list 3, 5, 7, 9?" -- and she answers every question correctly. What would you bet her Quant score will be on the real GMAT? Would you bet on Q51? It's of course possible the person is a Q51 level test taker, since any Q51 test taker would answer those easy Quant questions correctly. But so would a Q50 level test taker, or a Q49 test taker. And if you had to bet, you'd be right far more often if you bet on Q49 or Q50 than on Q51, just because so many fewer people are Q51 level to begin with.

Of course medium level Verbal questions are much, much harder than those Quant questions. But the V50 and V51 level are extraordinarily far above average -- test takers at that level are in the extreme upper reaches of the 99th percentile. A V50 level test taker is not supposed to find medium level (V30 level) questions at all challenging. Nor is a V51 level test taker. Both the V50 and the V51 level test taker will often get thirty of those V30-level questions right. But (if those thirty questions are easy enough) you might be right more often 'betting' the person doing that is a V50, just because there are more V50 level people in the world than V51 level people. And that's what the algorithm is doing - looking at your response pattern, and then working out probabilistic what your most likely ability level is.

That does seem unfair to me - it's not the OP's fault the test just didn't have enough hard questions to deliver. As bb points out, even designing a lot of questions that would let the test distinguish between V49, V50 and V51 level test takers is a nearly impossible task (it's much easier to design very hard Quant questions) and doing so wouldn't make a lot of sense from a resource allocation perspective, since they'd only ever want to use those questions for the minuscule fraction of test takers at the V49+ level. But it does seem that the OP's questions were unusually easy for someone getting every question right, and if they had been slightly harder throughout, the algorithm could have draw a distinction between V50 and V51.

There is one other possibility that occurs to me - the test has never been able to distinguish with a lot of statistical certainty between V50 and V51 level test takers, because the test doesn't ask hard enough Verbal questions to do that. So it's possible they've given up trying -- they might just be capping Verbal scores at V50 now instead of V51. That's purely a guess, and one I don't think is likely to be right, but when I google "V51 GMAT 2017" I'm not finding any debriefs from this year (in two seconds of searching) that report V51 scores. Mind you, V51 scores are exceedingly rare, so that's maybe what you'd expect regardless.

Lastly, congratulations on the outstanding score! I'll be curious to know what response you get from GMAC.

Thanks Ian!

I looked further into past debriefs after I made the original post and saw that Q51/V51 appears to be the only way to an 800 and that the charts I was looking at are outdated. Confusingly these outdated charts have been reposted in recent history through fairly official outlets...

Manhattan Prep seemingly has it right: https://www.manhattanprep.com/gmat/gmat ... alculator/

Beside all that, there is a debrief of Q51/V51 790 from 2016.

I accept that the difficulty of the questions I received may only warrant a 50, but if the differentiation between those verbal scores is how they issue the test it seems the difference between a 50 and 51 should be nil with regard to total score. It would be interesting if they no longer score the verbal section higher than a 50 and a Q51/V50 could be an 800... in which case dammit I knew I should have studied DS more! :p

Regardless what the total score is/how the mechanics should work (I did not even mention it when I reached out to GMAC), and the average difficulty of the questions, I still can't reconcile the verbal subcategory scores with the total verbal score. Per the ESR I got a 51 on each question category which in my mind necessarily implies a 51 total score. Looking at it the other way, with a score of 50, if 51 is still possible, there was some manner in which my performance could have been higher, but my subcategory performances implies there was no way for my performance to have been higher. I can't see a way for the total score to be accurate or the subcategory scores to be accurate without a contradiction.

I guess all this could all be explained by the scoring algorithm complexity and cost to accurately score 100% of test takers vs 99.99% of test takers not being warranted. Where resources are not limitless, a more complex scoring algorithm or more resources spent on generating tough verbal questions to differentiate at that level would detract from the purpose of the GMAT to characterize the mental horsepower relevant to business school for the population of MBA hopefuls.

Whatever the response from GMAC, I'm excited at the prospect of knowing where the contradiction lies, or if there is no contradiction in my verbal scoring, knowing what I am not understanding.

Congrats on the score

You can have your test re-scored, but I doubt the score will change and I doubt, even more, they will clue you in on why it wasn't a V51, because doing so would tip their hat into their scoring algorithm, which I am sure they want kept a secret.



Just for fun: a CR problem involving real information to help you clue in on why.

UCF is the only undefeated football program in college football, however they are ranked #14. Clearly the ranking system is incorrect, since UCF should be ranked #1.

Which of the following, if true, would weaken the conclusion?
(a) Although UCF is undefeated, they played the worst teams in the league and barely won those games.
(b) UCF has never had to travel more than 250 miles for an away game.
(c) UCF's stadium is one of the loudest and makes it hard for opponents offensives to call plays.
(d) UCF is the only team in college football without significant injuries
(e) UCF has more seniors on its roaster than any other team

Of course the correct answer is (a). Played the worst teams (you weren't given enough hard questions) and barely won those games (GMAT doesn't give style points: doesn't matter if you can solve an question using 1 method or 3 different methods or blindly guess the correct answer ....it's a right or wrong thing, so we can eliminate right/wrong answers from your possible reasons).

If you think about it, the only logically explanation is that you weren't given enough difficult questions. Now, why weren't you given enough difficult questions? I have two theories:

(1) You were dealt a bad hand of cards.
To use an extreme example, if I sat for the GMAT and was given basic arthritic questions (1+1=?), I am sure I'd get all the questions correct, as would most people; however, we both can agree that should not warrant a Q51. Ideally, yes, if you were answering all the questions correctly, you should have an opportunity for a V51; however, the GMAT may not be so ideal. Most standardized tests focus around the median score, and the accuracy of the middle percentiles, the extremes may not be perfectly skewed.

(2) Although the experiment questions are not "graded", they may affect the overall difficulty level of all the questions you see, and the overall difficulty level of the questions affects your score.
Essentially what I am saying is you missed a few of the experiment questions, which lowered the overall difficulty level of all questions presented to you, so even though were answering the non-experimentally questions correctly, they weren't the *most* difficult.

To test this theory, we can use reverse logic. I believe the difficulty level of your experiment questions are determined by your performance. I didn't score as well as you have; however, I can attest to the fact that I did not see any very easy questions. By very easy questions, I mean the first dozen or so questions that appears in the official guides.
This makes sense in that giving a sub-600 level experimental question to a 700+ scorer is essentially wasting an experiment question and vice versa... giving 700 level experimental questions to a clown is wasting an experiment question.



Nonetheless, a great score, I don't think any school will consider a 790 any differently from an 800.

Just read this now, I do like the theory of Q51 being graded differently for 0, 1, and 2 missed questions and this could be the explanation for 790...the quant score. Very good outside the box but feasible reasoning

However, I do disagree with the experiment questions being grabbed out of the air and appearing on the test. I am sure there are multiple checkpoints that need to be checked off before an experimental question appears on test even as an experimental question. When you create a drug, it takes on avg ~10 years to go to market. You don't create a drug and shove it down human test groups lol. The drug, the science behind it needs to be harm free first, then it's tested in rats, then more human like animals like monkeys, and finally as the last step, it's tested in humans. When experimental appear on the test, it's the last test. I think test takers know the general difficulty level of an experimental question. As mentioned before, I've never heard of a 750+ taker say he received very easy sub 600 level questions.
When experimental questions appear on a test, it's less of a should we keep it and more of an let's get some data about the difficulty level so we can better determine who to administer this question to.

Lastly, there is no one question to ask that would differently a 98th from an 99th percentile scorer. Doing so would just be a blind guess/random question "What was George Washington's shoe size?", also each response essentially has an 20% chance of being right. The way the test works is, they toss out very hard questions (less than 50% answered correctly), and toss out very easy questions (more than 90% answered correctly). And stick to questions with a base of around 60%-85% answered correctly. Essentially how you separate top scores is by asking enough of the 60% right hard questions.
(6/10)^8 = 0.01 = 1%
Meaning if you ask 8 questions that each have a 60% chance of getting right, then only 1% of test takers will be able to answer all 8 of these correctly.

As I pointed out in my earlier post, this is not true. Your experimental questions have no effect on the rest of your test. Before you even start your test, the computer selects a predetermined batch of experimental questions that you will see, and randomly determines where in the test you will see those questions. Nothing you do during the test can change that - which experimental questions you get and when you get them is all determined before you answer even one question.



Except where I mentioned I was speculating in my post, all of the information I was providing comes directly from GMAC research reports, and from academic articles about implementing IRT-based algorithms. I wasn't stating an opinion. It is definitely true that questions go through a rigorous editorial process before they are even used as experimental questions. But it is also true that every experimental question needs to be administered to test takers across the full ability spectrum. If you don't know about the details of IRT testing, the reasons won't make sense to you, but doing that is the only way to confirm the question has the characteristics an IRT test question needs to have, and it's the only way to compute the discrimination and pseudo-guessing parameters that the algorithm needs to use when the question becomes a real test question.



I didn't say there could be. The algorithm does indeed do what you describe, to distinguish between 98th and 99th percentile test takers. But the difference between a V50 and a V51 is not the difference between the 98th and 99th percentile. Depending on the year you look at, the 99th percentile in Verbal begins at the V45 or V46 score. At V50 and V51, you're very, very deep into the 99th percentile. You'd need to ask several hard questions to decide with any statistical certainty whether a test taker is a V50 or V51, because on a batch of medium and medium-hard questions, the performances of a V50 and of a V51 test taker will often be indistinguishable.

Ian, you seem to know a lot about the GMAT adaptive scoring algorithm. Could you please point us to the place in the literature where it tells us definitively that a test-taker's responses to the experimental questions on the GMAT don't affect the overall difficulty level of the questions? I'm not doubting you, but I can't verify that, either.

As far as I know, all GMAC has told us so far is that experimental questions are "unscored." However, technically a question could be unscored, but an incorrect answer could still produce a drop in the overall difficulty level and lead to scenarios such as this, where 100% of counted questions correct on Verbal inexplicably earns a less than perfect 50/51. This may or may not be true, but I think that the GMAT community at least deserves an answer from GMAC.

I do think it's possible that a 50 is perfect on the Verbal now, but there is not yet enough of a sample size to know for sure. So far I've earned a V48 pre-ESR, a 47 with 2 counted questions wrong, a 46 with 3 counted questions wrong, a 42 with 5 questions wrong and a 40 with 7 counted questions wrong.

To summarize (see ESRs below):

51 = 0 wrong
50 = 0 wrong (in the case of the OP and now another student as well) with perhaps an experimental question or two wrong
48 = 1 wrong
47 = 2 wrong
46 = 3 wrong
42 = 5 wrong
40 = 7 wrong (ran out of time at the end)

As I said in my previous post, my information comes from an official research report GMAC issued a few years ago that specifically discussed how experimental questions are used on the GMAT. That report was on their official website, in the research report section - if it's still online, that's where you'll find it.

This thing is bogus.

The OP got every scored verbal question right.

Experimental questions should not count at all, and I doubt that they do count, because, uh, they are experimental. So, they are NOT USEFUL IN DETERMINING SCORES.

The OP could do nothing more in verbal than get every question right.

It's pretty simple. If a test taker can get every non-experimental question right in the verbal section and not score V51, the test is flawed. There is no other possible conclusion.

GMAC has two things to do now.

- Admit that the test is flawed and fix it.

- Either give the OP V51 and 800, or offer the OP an opportunity to take the test again without paying the $250, AFTER THE TEST IS FIXED.

I agree! A test taker can't possibly perform any better than getting every counted question right--and if GMAC is telling us the truth, then the experimental responses should have zero effect on one's score. Thus, 100% of counted questions correct should yield a perfect score every single time. At the very least, the OP should have earned a Q51, V51, 790 composite--since his Quant score was "rounded up" to a perfect 51 with one DS question wrong.

The only rational explanation is that V50 is now a perfect score. If otherwise, then yes, the test is flawed, possibly because (as bb originally postulated) the experimental responses--though technically "unscored"--might still affect the overall difficulty level of the questions. The other possibility, of course, is that it's NOT flawed...that GMAC has been intentionally utilizing our responses to the experimental questions as determinants of overall difficulty level all along, but was hoping not to let that particular cat out of the bag.

In a weird way, though, this seems fair. When we take the real GMAT, 25.6% of the questions are experimental. We don't know which ones they are, however, so we must try our best on them. So shouldn't we be rewarded for getting the experimental questions correct in some minor way, even if it's just a sustained high difficulty level for the remainder of the section? Just my two cents.

For what it's worth, the GMAC website still indicates 51 as the highest possible Verbal score. Then again, it also indicates that scores range from 0-60 in the right-hand column, which we know to be untrue.

Hi all,

I am reporting back in with an unmomentous update.

I hadn't heard back from GMAC since they confirmed that they are investigating the test results, so I reached out to them last Friday. I got a reply back (Sunday evening no less), that they are still investigating the results and I'll hear back from them whenever they have made a conclusion. No timeline was given, but then again if the issue isn't fully characterized and they have to contend with holiday vacations and the inevitable end of year rush I am not surprised this will take some time.

I hope to know the outcome before Christmas and I plan to check back in with them in two weeks time to get a gauge whether or not that is likely.

Thanks to all who have commented recently and offered additional insight!

I'll be flattered if GMAC offers a free retake, but I don't see how I could avoid psyching myself out trying to beat my score. Like I mentioned in my first post, I believe that my mental state while taking the test of "I just want a 760 or higher, Im confident I can get that, let's see how high I can score" was key to me scoring as I did. If I go in thinking "This could be pointless unless I get every single question right" I think the probability of achieving the performance I did the first test will be low.

I agree; even a free retake would be unfair to you, because it's unlikely that you would get 100% of the scored Verbal questions correct twice in a row, and even 1 wrong bumps you down to 48/51 (see ESR a few posts above). Only 1 question wrong out of 58 counted questions is an amazing performance, and would be tough to replicate. You shouldn't have to be near-perfect twice in a row, just because GMAC messed up the Verbal scoring algorithm / question bank.

Plus, when you send your GMAT score to B-schools, they don't get to see your ESR--just your school score report. Despite answering one DS question incorrectly, you still received a perfect Q51. So as far as these admissions committees know, you answered 100% of the questions on the Quant section correctly (and it appears from your V50 that you either got one counted question wrong--even though you didn't--or that you perhaps got an experimental question or two wrong, since one counted question wrong usually yields a V48). And a Q51, V50, 790 score is still going to be better than 99.9% of the scores they see, even if GMAC doesn't ultimately decide to give you a perfect 800, or a Q51, V51, 790 composite.

For proof of this, check out the Harvard Business School Website, which indicates that the highest GMAT score from the class of 2019 was a 790 (but also confirms that it is in fact possible to score V51).

Wow, I had no idea that one question wrong is that impactful to the verbal score. At that score then it seems to be a game of "don't be human and let your attention waiver at any point".

That's a good point concerning what B-schools see and what they care about. I am under no pretense that 800 vs 790 will be the differentiator between admitted and not admitted anywhere. If I had to make a bet, I would bet it's impact on my probability of admittance is exactly zero since I can't imagine schools care that your attention was 100% laser focus instead of 99% on a test. That may be the difference between having the slight urge to pee the last couple questions of a section and having an empty bladder. I say that somewhat in jest, but I am a voracious water drinker so this was a legitimate worry for me .

It should be reiterated I did not reach out to GMAC because I think I deserve an 800; I reached out purely because I couldn't figure out how my scoring is coherent. I would walk away from this happy with only an explanation. But of course if I walk away from this with an 800 I'm going to have a stupid grin on my face for a couple days. Like everyone else, I'm just a monkey with shoes so of course I have daydreamed how awesomely snarky it would feel to say to people "They issued a revision to my gmat score. I didn't score a 790 *pause* I scored an 800". I would take that as my only Christmas present.

One wrong answer in Verbal usually produces a V49 or V50 score, and would only produce a V48 if the wrong answer were on an easier question (except in unusual circumstances, where the test was unusually easy overall). You can see that just by experimenting with GMATPrep tests, or from the fact that people report getting V49 scores on the real test.

I still don't think it's possible to get an 800 with a mistake in the Quant section (as the Q51/V51/790 score report I linked to above suggests) even if it's possible to get a Q51 with one mistake. So I think the real question is whether your Verbal score ought to have been a V51, and if they're still giving out V51 scores, I think it should be.

Theoretically, this is true, but after searching GMAT Club (and the rest of the internet) high and low, I see no concrete evidence that scores of V49 or V50 are currently possible with one counted question wrong. Could you please point me to an ESR that supports this assertion? If you look at my example of the V48 with one wrong ESR posted above, you will notice that the test-taker did not answer a question incorrectly until the final quarter of questions, which suggests that this question was of high difficulty, not low difficulty.

The GMATPrep software utilizes a different scoring system than does the real GMAT (no experimental questions on GMATPrep), so unfortunately that would be an "apples to oranges" comparison; thus the fact that you can reverse-engineer a score of V49 or V50 on GMATPrep is irrelevant.