Events & Promotions
|
|
GMAT Club Daily Prep
Thank you for using the timer - this advanced tool can estimate your performance and suggest more practice questions. We have subscribed you to Daily Prep Questions via email.
Customized
for You
Track
Your Progress
Practice
Pays
Not interested in getting valuable practice questions and articles delivered to your email? No problem, unsubscribe here.
May 1212:00 PM EDT
-11:59 PM EDT
Make the most of your break with the most realistic GMAT™ prep. Take up to $700 off select products.
May 1308:30 AM PDT
-09:30 AM PDT
In Episode 4 of our GMAT Ninja CR series, we tackle the most intimidating CR question type: Boldface & "Legalese" questions. If you've ever stared at an answer choice that reads, "The first is a consideration introduced to counter a position that...- May 14
08:30 AM PDT
-09:30 AM PDT
Most GMAT test-takers are intimidated by the hardest GMAT Verbal questions. In this session, Target Test Prep GMAT instructor Erika Tyler-John, a 100th percentile GMAT scorer, will show you how top scorers break down challenging Verbal questions..
25 Sep 2019, 20:10
Kudos
Bookmarks
I've been analyzing through many ESR reports (scores ranging from 480 to the upper 700s) on GMATclub, and this is what I observed:
For Quant: The average question difficulty appears to be medium-medium high (based from the ESR reports), depending on how the test maker is doing. Generally, if the test maker did well in the first quarter, the difficulty tends to jump from Medium to Medium-High and levels off throughout the 3rd and 4th quarter of the exam. While I know that missing an easy question, even on the later quarters on the exam, hurts your score more than missing a hard question, in order to even have a chance to get the "hard" questions he/she has to answer the easy/medium questions correctly. (I believe the ESR proves how important doing well in the first quarter is crucial for a high score, more information on that, check out https://gmatclub.com/forum/new-format-g ... 69682.html) Even for test-takers who score a Q49-Q51, the average difficulty per section doesn't change much, based on the ESR difficulty. (That will lead to a couple of questions later in this post)
For Verbal: The average question difficulty appears to be medium to medium-high on rare occasions. It appears that the GMAT tends to rate the average verbal questions difficulty less than the difficulty in the Quant Section. Maybe this is one of the reasons why you can generally miss more Quant questions and still score a +Q48 out of Q51, but missing a few question on Verbal drops a V51 to V40 or even less because the GMAT considers Verbal questions to be "easier" (based on the ESR reports) than Quant so you have a higher penalty for missing a question?
(If anyone has any observations/comments on the ESR difficulty, please feel free to share!!!)
Now I have a few questions:
1.) Based on the ESR difficulty, does the GMAT generally test you on mostly easy/medium questions, then occasionally test you on "hard" questions throughout the exam? Because if the GMAT gave a test-taker all bunch of "hard" questions after he/she did well in the first quarter of the exam, then I would assume the difficulty in those later quarters will be closer to high, and not medium-high (unless the GMAT underestimates the difficulty of the questions that are given??).
Similar to GMAT prep exam 1 and 2, generally the bulk is mostly easy/medium questions (based on GMATclub's difficulty), and if you're doing well in the first quarter, you'll still get "medium" questions, but the software will give you a few hard questions. However, exam 3-6, there are less hard questions in the question pack, so the average difficulty will be less (the exam will give you even more easy/medium question but even fewer harder questions than GMAT prep exam 1/2), but the test-taker will have less room for error because even a couple of wrong answers on easy/medium question damage your score. Based on the forums, the actual GMAT exam probably coincides more closely to GMATprep 1 and 2 because the GMAT will have a much larger question pool to give a test-taker harder questions (assuming the test taker is doing well). Maybe this is why people who are prepping for the GMAT should focus on fundamentals, then nailing the easier/medium questions from the OG guide rather than just practicing super-tough GMAT questions, because the bulk of the exam is those easy/medium questions? So if you don't have the consistency of answering those easy/medium questions, you'll never have the opportunity to attempt those "hard" questions.
2.) Or is the GMAT grossly underestimating the question difficulty on its ESR reports? Can anyone who scored well on either the quant or the verbal section (or both) give insight on how the question difficulty on the actual test compared to the difficulty from the OG guides and GMATprep questions difficulty on GMATclub?
For Quant: The average question difficulty appears to be medium-medium high (based from the ESR reports), depending on how the test maker is doing. Generally, if the test maker did well in the first quarter, the difficulty tends to jump from Medium to Medium-High and levels off throughout the 3rd and 4th quarter of the exam. While I know that missing an easy question, even on the later quarters on the exam, hurts your score more than missing a hard question, in order to even have a chance to get the "hard" questions he/she has to answer the easy/medium questions correctly. (I believe the ESR proves how important doing well in the first quarter is crucial for a high score, more information on that, check out https://gmatclub.com/forum/new-format-g ... 69682.html) Even for test-takers who score a Q49-Q51, the average difficulty per section doesn't change much, based on the ESR difficulty. (That will lead to a couple of questions later in this post)
For Verbal: The average question difficulty appears to be medium to medium-high on rare occasions. It appears that the GMAT tends to rate the average verbal questions difficulty less than the difficulty in the Quant Section. Maybe this is one of the reasons why you can generally miss more Quant questions and still score a +Q48 out of Q51, but missing a few question on Verbal drops a V51 to V40 or even less because the GMAT considers Verbal questions to be "easier" (based on the ESR reports) than Quant so you have a higher penalty for missing a question?
(If anyone has any observations/comments on the ESR difficulty, please feel free to share!!!)
Now I have a few questions:
1.) Based on the ESR difficulty, does the GMAT generally test you on mostly easy/medium questions, then occasionally test you on "hard" questions throughout the exam? Because if the GMAT gave a test-taker all bunch of "hard" questions after he/she did well in the first quarter of the exam, then I would assume the difficulty in those later quarters will be closer to high, and not medium-high (unless the GMAT underestimates the difficulty of the questions that are given??).
Similar to GMAT prep exam 1 and 2, generally the bulk is mostly easy/medium questions (based on GMATclub's difficulty), and if you're doing well in the first quarter, you'll still get "medium" questions, but the software will give you a few hard questions. However, exam 3-6, there are less hard questions in the question pack, so the average difficulty will be less (the exam will give you even more easy/medium question but even fewer harder questions than GMAT prep exam 1/2), but the test-taker will have less room for error because even a couple of wrong answers on easy/medium question damage your score. Based on the forums, the actual GMAT exam probably coincides more closely to GMATprep 1 and 2 because the GMAT will have a much larger question pool to give a test-taker harder questions (assuming the test taker is doing well). Maybe this is why people who are prepping for the GMAT should focus on fundamentals, then nailing the easier/medium questions from the OG guide rather than just practicing super-tough GMAT questions, because the bulk of the exam is those easy/medium questions? So if you don't have the consistency of answering those easy/medium questions, you'll never have the opportunity to attempt those "hard" questions.
2.) Or is the GMAT grossly underestimating the question difficulty on its ESR reports? Can anyone who scored well on either the quant or the verbal section (or both) give insight on how the question difficulty on the actual test compared to the difficulty from the OG guides and GMATprep questions difficulty on GMATclub?
25 Sep 2019, 22:55
Kudos
Bookmarks
I have to say you are very well informed and aware of the GMAT test background and elements (diff tests having a deeper/shallower pool). Kudos!
There is obviously a bit of a question as to why you are diving into this question and its practical application (I can tell you that there are probably 10 people on GMAT club such as IanStewart and myself included, who can spend hours talking about the theoretical aspects of the GMAT and not get bored) so I am totally comfortable leaving the practical application of this question tucked away for now.
I would generally agree with your analysis. I have noticed on my real test and also in the practice GMAT Prep tests that the difficulty is not constant and it fluctuates even if you are doing well. I have a number of theories about the GMAT, and they are all mostly about optimization and building a CAT test efficiently in such a way as to minimize the cost and effort of test writing since people can take the test every 15 days. Having to create a database of thousands or even hundreds of questions would be too burdensome and difficult. Moreover, creating a new database of HARD only questions for those test-takers who can score a V51 and a Q51, would be even more wasteful since there is just 1% of them. The main mass is in the middle of course and thus the best bang for the buck would be writing as few diff questions as possible and recycling them as much as possible (let's touch recycling as that will take a whole separate page and that's not the point of this discussion).
So, as you very aptly pointed out, a test with few hard questions and generally a shallow pool of questions will serve a good test-taker mostly medium diff questions but will require a larger number of correct answers than a test with mostly difficult questions. Thus GMAC has a few ways of dealing with high scorers - they can control the pool of questions and give them hard questions or they can use a shallow pool but be much more punitive with mistakes. (I am not saying this quite perfectly but I think you understand what I mean). There are many levers available to test-creators and also having experimental questions that are not graded is an additional layer of obscurity that makes some of these evaluations harder and understanding one's performance during the test potentially a pointless task (which smart people say take it one question at a time).
Easy vs. Diff Questions.
To start answering your questions, you are correct - being good about solving easy/medium questions is the path to the high score. People mistakenly chase hard questions, thinking that if they can solve them, they are guaranteed a high score, only not to see more than 1-2 of them on the test.
Quant:
I had only one hard question (which i failed to answer correctly) on my test. It was a probability question which was not that hard now that i look at it, but I was not prepared for probability. I spent 3 mins on it, and ended up guessing wrong, and move on. I got Q49 despite that wrong guess, just 2 points short of the max score. All the rest of my questions were medium and easy diff on the test. I clearly made more than than one mistake but in my opinion none of the questions I was tackling were hard... though twice! i solved a question only to see that my answer was not listed among the answer choices. Twice on the real test i was lucky not to have a match. I messed up up under pressure. Everyone does and mostly on easy questions with just simple gotchas. If you can be good at those and not caught caught up in the panic, you can get at least Q49 without ever touching a hard Quant question - I clearly have proved it on my test.
Verbal:
I got V42, which is still 96th percentile. It is not V47 or something super high, but in terms of difficulty, you are almost at the top already, so likely I have made a few more mistakes than I should have (unfortunately no ESR in my days). I can't tell you if the questions were particularly hard - it was really hard to judge the difficulty level because I felt I was guessing every question. I was just barely certain in my answers and if I had the time, I would spent hours on each question but I had to pick an answer and move on, which I did. For about half the verbal section, I felt I was failing. I had no confidence in my choices/answers, they just seemed slightly better than the others but not necessarily right. Many people who score well on Verbal report that they felt as though they were guessing, which those who get low Verbal scores, often report they felt very good about their answers and were pretty certain.
Difficulty in General:
I looked up the hardest and easiest questions on GMAT Club and found that there are few Official Questions in the Top 100 and quite a few in the Easiest 100. That's 100 out of thousands of questions here, so in general the difficulty of the official questions is not high, which confirms the concept that GMAT probably does not have many hard questions and difficulty sort of bobs around medium/hard/medium for even the top scoring individuals, likely because of the question pool lacking sufficient number of the hard questions.
https://gmatclub.com/forum/150-hardest- ... 04134.html
https://gmatclub.com/forum/150-hardest- ... 04136.html
I assume you have seen this big back and forth discussion that is quite interesting: https://gmatclub.com/forum/q51-v50-253258.html It talks a bit about one of your questions related to Verbal Difficulty and how could a person with all answers correct get only V50? (their score got updated to V51, but it stayed at 790).
There is obviously a bit of a question as to why you are diving into this question and its practical application (I can tell you that there are probably 10 people on GMAT club such as IanStewart and myself included, who can spend hours talking about the theoretical aspects of the GMAT and not get bored) so I am totally comfortable leaving the practical application of this question tucked away for now.
I would generally agree with your analysis. I have noticed on my real test and also in the practice GMAT Prep tests that the difficulty is not constant and it fluctuates even if you are doing well. I have a number of theories about the GMAT, and they are all mostly about optimization and building a CAT test efficiently in such a way as to minimize the cost and effort of test writing since people can take the test every 15 days. Having to create a database of thousands or even hundreds of questions would be too burdensome and difficult. Moreover, creating a new database of HARD only questions for those test-takers who can score a V51 and a Q51, would be even more wasteful since there is just 1% of them. The main mass is in the middle of course and thus the best bang for the buck would be writing as few diff questions as possible and recycling them as much as possible (let's touch recycling as that will take a whole separate page and that's not the point of this discussion).
So, as you very aptly pointed out, a test with few hard questions and generally a shallow pool of questions will serve a good test-taker mostly medium diff questions but will require a larger number of correct answers than a test with mostly difficult questions. Thus GMAC has a few ways of dealing with high scorers - they can control the pool of questions and give them hard questions or they can use a shallow pool but be much more punitive with mistakes. (I am not saying this quite perfectly but I think you understand what I mean). There are many levers available to test-creators and also having experimental questions that are not graded is an additional layer of obscurity that makes some of these evaluations harder and understanding one's performance during the test potentially a pointless task (which smart people say take it one question at a time).
Easy vs. Diff Questions.
To start answering your questions, you are correct - being good about solving easy/medium questions is the path to the high score. People mistakenly chase hard questions, thinking that if they can solve them, they are guaranteed a high score, only not to see more than 1-2 of them on the test.
Quant:
I had only one hard question (which i failed to answer correctly) on my test. It was a probability question which was not that hard now that i look at it, but I was not prepared for probability. I spent 3 mins on it, and ended up guessing wrong, and move on. I got Q49 despite that wrong guess, just 2 points short of the max score. All the rest of my questions were medium and easy diff on the test. I clearly made more than than one mistake but in my opinion none of the questions I was tackling were hard... though twice! i solved a question only to see that my answer was not listed among the answer choices. Twice on the real test i was lucky not to have a match. I messed up up under pressure. Everyone does and mostly on easy questions with just simple gotchas. If you can be good at those and not caught caught up in the panic, you can get at least Q49 without ever touching a hard Quant question - I clearly have proved it on my test.
Verbal:
I got V42, which is still 96th percentile. It is not V47 or something super high, but in terms of difficulty, you are almost at the top already, so likely I have made a few more mistakes than I should have (unfortunately no ESR in my days). I can't tell you if the questions were particularly hard - it was really hard to judge the difficulty level because I felt I was guessing every question. I was just barely certain in my answers and if I had the time, I would spent hours on each question but I had to pick an answer and move on, which I did. For about half the verbal section, I felt I was failing. I had no confidence in my choices/answers, they just seemed slightly better than the others but not necessarily right. Many people who score well on Verbal report that they felt as though they were guessing, which those who get low Verbal scores, often report they felt very good about their answers and were pretty certain.
Difficulty in General:
I looked up the hardest and easiest questions on GMAT Club and found that there are few Official Questions in the Top 100 and quite a few in the Easiest 100. That's 100 out of thousands of questions here, so in general the difficulty of the official questions is not high, which confirms the concept that GMAT probably does not have many hard questions and difficulty sort of bobs around medium/hard/medium for even the top scoring individuals, likely because of the question pool lacking sufficient number of the hard questions.
https://gmatclub.com/forum/150-hardest- ... 04134.html
https://gmatclub.com/forum/150-hardest- ... 04136.html
I assume you have seen this big back and forth discussion that is quite interesting: https://gmatclub.com/forum/q51-v50-253258.html It talks a bit about one of your questions related to Verbal Difficulty and how could a person with all answers correct get only V50? (their score got updated to V51, but it stayed at 790).
26 Sep 2019, 15:55
Kudos
Bookmarks
I'm in the process of moving, so my apologies to anyone who has mentioned me recently (or who has PMed me) and hasn't received a reply!
A couple of comments about the posts above:
- It's much easier to design high-quality and very hard questions in Quant than in Verbal. I could easily design a Quant question that even 800-level test takers would have difficulty answering, but I'm not even sure what a Verbal question of that difficulty level could look like. So it should be true that the average difficulty level of hard Quant questions is higher than for hard Verbal questions, on the real test (though perhaps not on the paid-for tests, which have a limited question pool). But when discussing scoring for Quant and Verbal, it's important to recognize that GMAT scoring is based on how many standard deviations you are above or below average. In percentile terms, a Q50 is not even close to a V50 -- there's no Quant score that corresponds to the Verbal scores above V45. If you are thinking about the impact wrong answers can have on scoring, to make a fair comparison you should compare Q50 scorers with V40 scorers or thereabouts. A single wrong answer has a large effect on scores around the V50 level, but you don't see that effect in Quant because there is no analog of the V50 level on the Quant scale - there are not different Quant scores that differentiate between test takers deep into the 99th percentile.
- about interpreting average difficulty level from an ESR, it's impossible to say, from the ESR graphics, just how hard the questions are since there's no indication precisely what the vertical axis means. Anyone who has scored in the Q50-Q51 range or V45+ range on the real test knows that there are typically very many extremely hard questions at that level. But I've never seen an ESR where the average difficulty line reached even the top quarter of the available space. So I think the scale of those graphics is misleading - it seems to suggest that question difficulty doesn't deviate much from medium-level, but only because the top of the graph is probably for 900-level questions that don't exist in practice. That's my assumption, at least, but there's no way to confirm that.
- I don't think this is true. I recently went through a lot of the "hardest" Verbal CR questions in that list, and almost all of the "hardest" prep company Verbal questions are questions that are deeply problematic, logically speaking. When a question has two perfectly correct answers, or no logically correct answer, naturally test taker performance on that question will be poor, but not because the question is difficult, but instead because the question is poorly designed. One thing you can infer from that: it's difficult to design very hard Verbal questions. But I don't think you can infer anything about the difficulty level of questions on the real test from the fact that prep companies have designed some low quality questions that confuse a lot of people (myself included!). It's also clearly true that many prep company Quant questions are much harder than any real test questions, only because those company questions are overly long or complicated, or are out of scope. Designing extremely hard Quant questions is actually very easy to do, and the fact that some companies do it doesn't really tell you anything about how hard the GMAT is. Anyone scoring Q50-Q51 on the real test will observe that real GMAT Quant questions can be very hard (though not because they're extremely long or complicated or out of scope).
A couple of comments about the posts above:
jsound996
- It's much easier to design high-quality and very hard questions in Quant than in Verbal. I could easily design a Quant question that even 800-level test takers would have difficulty answering, but I'm not even sure what a Verbal question of that difficulty level could look like. So it should be true that the average difficulty level of hard Quant questions is higher than for hard Verbal questions, on the real test (though perhaps not on the paid-for tests, which have a limited question pool). But when discussing scoring for Quant and Verbal, it's important to recognize that GMAT scoring is based on how many standard deviations you are above or below average. In percentile terms, a Q50 is not even close to a V50 -- there's no Quant score that corresponds to the Verbal scores above V45. If you are thinking about the impact wrong answers can have on scoring, to make a fair comparison you should compare Q50 scorers with V40 scorers or thereabouts. A single wrong answer has a large effect on scores around the V50 level, but you don't see that effect in Quant because there is no analog of the V50 level on the Quant scale - there are not different Quant scores that differentiate between test takers deep into the 99th percentile.
- about interpreting average difficulty level from an ESR, it's impossible to say, from the ESR graphics, just how hard the questions are since there's no indication precisely what the vertical axis means. Anyone who has scored in the Q50-Q51 range or V45+ range on the real test knows that there are typically very many extremely hard questions at that level. But I've never seen an ESR where the average difficulty line reached even the top quarter of the available space. So I think the scale of those graphics is misleading - it seems to suggest that question difficulty doesn't deviate much from medium-level, but only because the top of the graph is probably for 900-level questions that don't exist in practice. That's my assumption, at least, but there's no way to confirm that.
bb
- I don't think this is true. I recently went through a lot of the "hardest" Verbal CR questions in that list, and almost all of the "hardest" prep company Verbal questions are questions that are deeply problematic, logically speaking. When a question has two perfectly correct answers, or no logically correct answer, naturally test taker performance on that question will be poor, but not because the question is difficult, but instead because the question is poorly designed. One thing you can infer from that: it's difficult to design very hard Verbal questions. But I don't think you can infer anything about the difficulty level of questions on the real test from the fact that prep companies have designed some low quality questions that confuse a lot of people (myself included!). It's also clearly true that many prep company Quant questions are much harder than any real test questions, only because those company questions are overly long or complicated, or are out of scope. Designing extremely hard Quant questions is actually very easy to do, and the fact that some companies do it doesn't really tell you anything about how hard the GMAT is. Anyone scoring Q50-Q51 on the real test will observe that real GMAT Quant questions can be very hard (though not because they're extremely long or complicated or out of scope).
