I got 20 correct in the quant section yet ended up with 81 percentile

­Rajat, do you really think the behavior is problematic? I think it's logical that, once the test adapts, you get a lower score than you would have if you had gotten the early questions correct, even if you edit them.

The only issue I've seen is that, if you edit the early questions to correct, you get a perfect score, an outcome that doesn't make sense since the difficulty was adapted down as a result of your missing early questions.

So, by 81st percentile, do you mean to say that you got a Q83?

That does seem very strange that GMAT would penalize you so much for just one incorrect question.

Has GMAT recorded this somewhere that candidates would be penalized for marking questions incorrect initially and then changing them to correct responses?

Looking to hear from others.

­Ashish, doesn't it make complete sense that you'd score lower with edited answers than with originally correct answers?

The test is adaptive. Missing questions makes it easier. So, if you miss early questions and later edit your answers, you've seen a relatively easy test. Accordingly, it would be unfair for a test with edited answers to be scored as high as a test with originally correct answers.

So, you're not "penalized for marking questions incorrect initially and then changing them to correct responses." You just don't score very high because you drove down the difficulty of the questions.

 

­Unfortunately it does not make sense to me Marty :?.

So, basically if I get all 21 questions correct in Quant, but I get (say) 2 of them correct after initially marking them wrong, I would not be getting Q90.

Did I understand you correctly?

­Basically, yes.

And, it makes sense that you would not get Q90 in that case because you didn't see hard questions. You just got a set of relatively easy questions correct. So, you didn't achieve top performance.

What makes sense is that you'd get Q90 only if you got a set of relatively hard questions correct. For instance, if you got all questions correct the first time through, you'd get Q90. Or possibly, if you initially missed a question toward the end, say question 19, and still got hard questions all the way through the section, and then corrected your answer to that question, you'd get Q90.

On the other hand, if you miss questions 2, 4, and 5 and see easy questions for most of the test, then regardless of whether you fixed those answers, it would not make sense for you to get Q90 since your test would be much easier than that of someone who got every question correct the first time through.

Also, consider this. After you miss 2 and 4, question 5 is likely a relatively easy question. So, if you then correct 2 and 4 but not 5, you have a missed easy question in your set. So, of course, you would not score as high as someone else who had just one missed hard question.

All that said, from what I understand, all test-takers who have edited all answers to correct have scored Q90, but I find that pattern to indicate a bug in the test since those outcomes are not logical.

­This is frankly exactly how I would expect the test to behave. If a candidate has gotten all 21 questions correct, she should get a Q90, irrespective of whether she got (say) 2 of them correct, after getting them wrong initially. If the test does not behave this way, it should be clearly documented by GMAC. On a lighter note, it would just be absurd to look at the ESR for a candidate showing green ticks for all 21 questions, but getting (say) a Q87  .

The ability to change responses to three responses for each section, is a tool that GMAT has supposedly given to the candidates, to give them more control over how they approach the test. However, if the GMAT algorithm "penalizes" (I would still call it that) candidates for getting the questions correct by way of editing, I would not exactly call it much control/empowerment.

On a side note, candidates don't always get a question incorrect because they did not know how to solve it; we have been suggesting our candidates to use this ability (to edit responses) in order to manage their time more effectively during the exam. So, consider the following two candidates,  both of whom ultimately got all 21 responses correct:

Candidate-1: She finds Question 5 difficult, but still decides to spend an extraordinary 6 minutes on that question. She ultimately does get this question correct and finally somehow manages to finish the section on time.

Candidate-2: She also finds Question 5 difficult and so, rather than spending 6 minutes on that question (and risking not able to attempt the latter questions in the section due to lack of time), chooses to randomly mark a response to this question and move forward. Later, once she finishes all the questions in the section, she revisits Question 5 and devotes whatever time that she has left for the section, ultimately getting that question correct as well.

To think of it that Candidate-2 would get lesser marks than Candidate-1, for better time management, just doesn't seem fair to me.

MartyMurray - purely in ML estimate, you cannot give the entity, the flexibility to change the answer, and still penalize the entity (i.e. lower the difficulty level) for initially marking the incorrect answer. Kind of defeats the whole purpose (of giving the flexibility). Ideally, these questions should be treated like experimental questions - the difficulty level of subsequent questions served should not be changed based on the student's response to these questions. Then towards the end, when the user submits the final response, these questions should be evaluated, and the estimates updated. Otherwise, all you are telling the student is - we gave you the flexibility.. which you utilized and we penalized you for it.

Doing so (recalculating the estimates) requires either an iterative calculation or a large lookup table - both in today's world are not uber expensive (less than 10 cents) from a computation standpoint.

I have seen this happen so frequently (20/21 questions correct, and yet low scores) that I had to report this to the GMAC. I rarely saw such behavior in GCE but in GFE, this is prevalent (1/100 ESRs that I have seen).

-Rajat

­That doesn't make sense to me.

The only way to make the test adaptive is to reduce the difficulty of the questions after a question is missed, and as soon as you reduce the difficulty, you reduce the performance demonstrated by the test-taker.

In that case, the test-taker who has missed and then edited questions has not demonstrated the same performance as a test-taker who has gotten all the questions correct in the first place.

So, there's no way to "recalculate" at that point, and no need to. The test-taker saw a relatively easy test and got a score that fits that test.

The only thing GMAC has to do is make clear to test-takers that editing questions is not as good as getting them correct the first time.

If edit is possible and doesn’t impact score
I would just randomly mark 5-6 questions as wrong initially, drive the difficulty down and then come back and correct it. Making me get all correct and Q90 right? Is this possible?

Posted from my mobile device

­The issue is that missing questions and later editing them is not just better time management. It's weaker performance.

In your example, Candidate 2 saw an easier test.

What's unfair is for both candidates to get the same score.

And I think the key thing is for people to understand that.

I think what's going on is that people got it into their heads, possibly because of things said by GMAC, that editing answers would be the same as getting questions correct the first time, which is illogical, and now are surprised that it's not true.

The test is adaptive. Missing questions changes the difficulty of the test. Easier tests should generate lower scores.

It's nice that you can edit answers, but on a question-adaptive test, it's patently illogical for a test with edited answers and a test with initially correct answers to be awarded the same score. It's both illogical and unfair.

So, GMAC should make this fact clear, and test-takers should proceed accordingly.­

­You can edit only three questions.

That said, if I understand correctly, if you edit all to correct, you can score Q90.

At the same time, if you don't edit all to correct, then you will score lower than you would have if you had gotten them correct initially.

So, basically, the scoring system is a little distorted.

How is this not true? As you mentioned in your last post:

So, perhaps what OP is seeing is actually an anomaly that GMAC needs to fix.
 

­I think the fact that all test-takers who have edited all answers to correct have scored Q90 is the bug, or anomaly. The scoring is logical otherwise.

How is that possible? Did you edit the answers in the end?

That's strange. Pls raise it to GMAC

­Hi Banmeet,

I'm not a quant expert, but I've been following reports like yours closely. As the others have mentioned, this was most likely because the difficulty level went down significantly at the beginning of your test.

If that is what happened (and we can't be sure that it is), I'd say it's unfair. Please do let GMAC know about this.­

Hi All,


I understand what marty is saying, but don't you think in renders the entire point of revisiting and correcting a mistake useless, as my first time answers are driving the test and the score and irrespective of whether I correct them or not will make little to no difference.

My another point on the same is that, I missed my questions till 4, beyond which I have taken less time and made none wrong, yet i did not get a hard or a very hard questions, if the test as it says is adaptive, it should have presented one of the hard questions after another 6-8 questions. but that sadly did not happen. Also, does time play a role in the increase of difficulty level?