GMAT Scoring Algorithm - My observations

There are still people reading this thread. I also have a thought regarding the admissions value of the GMAT.

Business Schools want diversity; it's all the rage. But of course that means accepting students from all parts of the country and world, with a wide range of background experience. So how does an Admissions officer decide whether an 3.45 GPA from Podunk College is really better than a 3.20 from Bombay Academy, or a 3.95 from Neverhearduvit? And how to balance resumes? The answer, very very often, lies in the GMAT.

The GMAT, after all, is a standardized test, which a lot of schools love for its 'objective' appearance and universal applicability. As a result, a good GMAT score can go far to make the difference, especially if (like me) you are a student with less than a sparkling GPA, or if (unlike me) your resume is a bit thin.

Never underestimate the impact of a standardized test.

OK here is my advice to all those who are following the strategy of “Concentrate First 25-Guess Restâ€

Great job!

Yeah, I think the algorithm used in GMAT is quite complex. I am not sure abt the first 20 correct theory either. To be on the safe side, just make sure that u focus on all the questions!

Haddy,

Thanks for the post, I was about to initiate same kind of post, just a bit different format. But I guess I should continue from where you left. My Idea is to get enough of score data in Excel sheet and analyze it. I will update detail of this spreadsheet in day of so.

I hope that few of us will post their score breakdown here and we can really strategize against the ‘Intelligent Design’ algorithm to increase our odds.

Chirag

I have to agree - the first 20 questions arent as important as once thought, and the first 10 sure arent. I find its really about getting bunches in a row correct and also finishing strong. It is not based on how well you start the test.

Regardless of first 20-25 question rule, is it still the case that earlier questions are weighted more, or is the algorithm adaptive to how you are doing. Seems like your score is hurt more when you get questions wrong in a row. I am really disappointed by the companies like kaplan and prencton, who think they know the algorithm, but it seems like they don't.

Thanks for your post haddy, I was really mislead by that first 25 questions 'rule'.

According to Princeton Review and Kaplan, you should invest in early few questions because they are heavily weighted.

According to Manhattan GMAT, GMAT is an adaptive test and it’s design such that it will be hard for brightest to answer first few questions correctly! So you should not preoccupy yourself with solving every question. And it is extremely important to complete test.

So, conflicting information, what to do?

One simple solution to gather lot of data from real software and see how it behaves, I have created simple Excel file. I would request all patron of club to take 10 Min and fill out this file with your GMATPrep practice score and email it to me, I will compile the data and results will be available for every one to analyze.

Purpose of this Drill:

Hope the data analysis will answer question such as..
- Are first few questions that crucial?
- Can one score high with couple of initial glitches?
- Do we penalize more for consecutive incorrect answer choices?
- What happens if you don’t finish test and guess last few?
- Is there relationship between score and number of correct answers?
- Is there relationship between subsections for example if you really bomb RC and perfect CR?

In short to have better understanding of how algorithm behaves..

--------
Other Reference on Club

https://www.gmatclub.com/phpbb/viewtopic.php?t=28493

I have created new thread under GMAT strategy and uploaded files

Agreed abt reverse engineering.

Its a java application, can easily run it through a dissasembler and get teh source code and then see whats the darn algorithm is!

Caveats thought:

1. I am not sure whether its legal or not and whether you can be sued for doing it!

2. I am not sure whether the algorithm in the practice test actually resembles the actual test! However, as the scoring is supposed to be somewhat similar, I wud assume that it (the algorithm) is similar!

What are the millions of Engineers doing? High time to decode the unknown!! If we could have decoded the genetic code... GMAT shud be peanuts!!

Disassembling is a great idea, though I doubt weather we can get much information from it due to complex nature of algorithm! More over it will be very tedious and perhaps unrealistic task in short timeframe and with available man power.

What we really need is general behavior and higher level observations about algorithm. This may be very effectively achieved with help of adequate data points. And I believe it’s more practical and realistic approach, not to mention legal..

Quotes from OG-11th Edition, Page 17

Myth: It is more important to respond correctly to the test questions than it to finish the test.
Fact: There is a sever penalty for not completing the GMTA test

If you are stumped by a question, give it your best guess and move on. If you guess incorrectly, the computer program will likely give you an easier question, which you are likely to answer correctly, and the computer will rapidly return to giving you questions matched to your ability. If you don’t finish the test your score will be reduced greatly. Falling to answer five verbal questions, for example could reduce a person’s score from the 91st percentile to 77th percentile. Pacing is important.

--------------------
My Analysis:
According to Appendix A, Table 1
91st percentile : Raw Score 40
77th percentile : Raw Score 35


So if you consider worst case scenario, where all five guess were incorrect, you can conclude that 1 point per penalty incorrect answer in final few questions for this particular poor candidate.[/quote]

Quotes from OG-11th Edition, Page 17
Myth- The first 10 questions are critical and you should invest the most time on those
Fact: All questions count.

It is true that the computer-adaptive testing algorithm uses the first 10 questions to obtain an initial estimate of your ability; however, that is only an initial estimate. As you continue to answer questions, the algorithm self-corrects by computing an updated estimate on the bases of all the questions you have taken, and then administers items that are closely matched to this new estimate of your ability. Your final score is based on all your responses and considers the difficulty of all the questions you answered. Taking additional time on the first 10 questions will not game the system and can hurt your ability to finish the test.

-------------------
My analysis:
Read between the lines…. Their (GMAC) Answer to Myth is “All questions countâ€

GMAC ultimately wants one to believe that all questions are important--and infact all questions are improtant if one has to achieve 800 score, but Let’s analyze above two Myths-Facts information available in OG

A) Last few Questions are Important
B) First few Questions are Important

Another fact is “there are few experimental questions in examsâ€

Naïve perhaps not so Naïve
--------------------------------

I strongly believe that concentration of experimental question should be lowest in initial few questions followed by last few and concentration should be heavier in middle section, why?

Think about person who is developing this algorithm!
He has tough job to do, he needs to
A) Accurately find real caliber of candidate
B) Also accurately judge level of experimental questions
(People who don’t know about judging level of questions visit…
https://www.manhattangmat.com/verbal4.cfm )

So now big question for his is how to intermingle experimental questions with real question? Simply he can equally spread this question across the test, but there may be a problem with this approach!

Initially he does not really know what is caliber of the candidate. So he has no idea which question to throw! Lets say if he gives 400 level experimental question to a person with 800 caliber that guy is going to solve it in few second giving him unfair time advantage. And if he throws 700 level experimental question to a 400 level candidate that poor guy will struggle and off course it will be unfair to the candidate.
So algorithm developer has no choice but to wait for some question still he has some idea about candidate.

Now let’s analyze what happens if algorithm developer gives last few questions as experimental questions! Simple, lot of people are just going to guess it! And that’s not what he wants! If people guess he will never know exact caliber of that questions! More over he needs to exactly know who reached last question in time or not, for example is last question is experimental how can he differentiate between candidate who reached last question and solved it rather than just guessed it!

So conclusively best place to throw experimental questions are middle section, now how do I define middle section! At present I don’t know, I would guess it’s very crucial to do first 10-15 question appropriately and also solve last few may be 5-10!

Some other anecdotal evidence for about above theory!
A) Read previous message, ETS emphasis that first few questions are used to acquire initial estimate of candidate’s caliber. And it also gives example where a person who went from 91% to 77% when he guessed it!
B) Lot of Test Companies are emphasizing on solving initial few questions carefully! When Kaplan and PR is writing something there might be some logic or anecdotal results behind this.
C) If you read briefing of some GMAT taker few suspects that guessing last few questions may have reduced their score!

So all above logic warrant conclusion that “concentrate on first few questions, speed up and finish last few questionsâ€

If someone is wondering why everyone suggests that you should guess last few questions rather than skipping them!

Simple lets say if you skip last 5 question, algorithm has to assume that all five were wrong and penalize you for that, but if you guess all 5, by simple probability one of that may be correct earning you some points, and no point is better than some..