bb wrote:
It is not possible to determine if scoring is correct or incorrect just based on pure number of questions but I can see that the student has had a horrible performance on the first 10 questions on the first test. There are experimental questions that don't count for example and those could throw things off more so. There is another project a user started looking at the difficulty of the questions a person answered:
https://gmatclub.com/forum/data-gmat-ev ... 73092.html - identifying the number of easy, medium, and diff questions can help.
At this time we do not have any indication that the GMAT Prep Scores are not accurate or cannot be trusted though I see there is another similar question here:
https://gmatclub.com/forum/gmat-prep-of ... 73060.htmlHas your student taken other trusted CATs and scored a lot better?
Do they want to take Old GMAT Prep or perhaps here is an idea for you - take a GMAT prep (old one) and make the exact same mistakes - e.g. miss the first 3, then a few correct, and then get the following wrong. Follow their footprint basically. The last 6 questions - don't even answer them and exist the test (we can back the score out).
Thanks, I'll try taking an old GMATPrep test and compare results. One thing I wanted to clarify is that I have dozens of students and have only noticed this issue in these two cases so I'm not trying to suggest there is a systemic error in everyone's score reports. I really am just trying to figure out how to explain these two results.
The poor performance in the first 10 questions could be the answer and is the most supported explanation at this time based on the link you posted. I'm not entirely convinced though.
For student #1: Yes, he gets half of the first 10 wrong. That's quite a bit less extreme than getting all of the first 10 wrong though like in the other post. Starting at question #16 he gets 14 in a row correct. Even if it's true that by this point the test has adjusted to the test-taker, and the difficulty level shouldn't fluctuate much from question to question, it just seems like it's not adapting at all which is bizarre. What would be the logic behind designing the test this way?
The best logical case I can make is by comparing the two students to each other. Student #2 gets fewer questions wrong but his percentile score is worse. As you have mentioned, it is impossible to explain someone's score based on a number of wrong answers alone. However, I would say that in addition to having fewer wrong answers, student #2's distribution is also clearly more favorable. He ends up getting 7 of the first 10 right despite getting the first three wrong. The only way I could see this being substantially worse than 5 out of 10 would be if those first three questions were so important that they doomed him for the rest of the section. After that, he gets some right and some wrong without any real streaks in one direction before finishing by getting 9 correct answers in a row. I just can't imagine the scenario where this results in a 28th percentile because it would have to mean he is getting absolutely creamed for every incorrect answer, and then getting almost no credit for his right answers. Really seems like it simply stopped adapting.