GMAC annual summit 2014

Question

Key Takeaways from the GMAT Summit 2014 NY 
GMAT Club attended the bi annual GMAT Summit in NY last week. The major topics of discussions were demographic changes, IR studies and the launch of the diagnostic score reporting. I will outline the takeaways in this post and outline the sessions overview in the next post. The Summit was attended by almost all major test prep companies, Wiley (partnering with GMAC to create the official guides), admission consultants, and representatives from GMAC.

Takeaways

Diagnostic Score Reporting  - GMAC announced the new feature of a diagnostic score preview. A report (if opted for, at a price) will provide more detailed percentile scores of quant and verbal sections (think algebra, geometry, SC, CR etc).

Diagnostic score report is still in its "beta" stage, and final report may be different from what we were shown in the presentation. 
 Score report will contain sectional percentiles including parts such as SC, CR, Algebra, Geometry etc. This will help test takers to understand weaknesses in terms of retake preparation. 
 Question by question analysis will not be available as of now. GMAC will also not disclose any numeric data (for example: number of questions correct or the likes). 
 GMAC is committed to improve the initial reporting and wants to make sure that the price of the report is well justified. 
 Diagnostic report of a cancelled score will also be made available on request. 
 So far diagnostic score report can be ordered for tests dating back to Oct 10th (or in that ballpark) for a price of $29.95 (subject to change) 
 GMAC also allows room for feedback and user suggestions to improve reports a continuous process of improvement

GMAC loves its Integrated Reasoning  -   IR is suggested to be one of the key differentiator in GMAT in the future with comparable Q/V scores. 
 GMAC has conducted validity research in 15 B School programs to figure out correlation studies with IR and B School GPA. Correlations suggest a strong relationship of IR Score and GPA. 
 GMAC has also been in a second validity study with a consulting company to relate IR trends with recruiting demands. GMAC suggested recruitment trends will align with IR scores similarly. Probably an interesting study for the next summit to reveal. 
 Bottom line is that IR is definitely going to get more important in the near future. So it cannot be the last week GMAT portion anymore, and probably deserve as much devotion as other sections of the test.

Vetting questions is incredibly scientific  - What decides whether an experimental item (item being the GMAC slang for a question) should migrate to the live test pool is based on quantitative and rigorous user data and subjective item analysis. 
  Quantitative user data involves reading graphs from the responses of GMAT Takers of different abilities (abilities defined in terms of the GMAT score). If the projected responses do not meet a specific response, the experimental question is rejected. 
 Subjective analysis makes sure that the question is free of demographic and other cultural biases. GMAC has been working on it already with fewer idiomatic usages tested in the last few years.

Possible new products and interaction space  - GMAC is also committed to create new products for test takers to help study for the test. Summit involved discussing the IR Prep tool, the Question and Exam Pack 1 and the effectiveness of those products in improving test scores. GMAC is also playing with the idea of having a microsite for test prep companies to collaborate and interact with the test makers in an open space. It has been made very clear that GMAC is open to feedback, so it's very useful to brainstorm ideas for better prep, practice and scoring, and as trends show GMAC will implement them to a reasonable extent. This was the basic theme of the summit, and it was very useful in terms of future updates being made more user centric.

Final thoughts on the GMAC summit and my impression  - I loved being in the summit, interacting with the amazing people from test prep companies, discussing what has worked and what hasn't in a GMAT test taker POV. I loved the general transparency in the room and especially from GMAC. All our questions were respectfully answered and there was an air of collaborative and collective action in the entire summit which I loved.

I got to talk to  VeritasPrepBrian , and yes: he is as amazing in person as he is on the videos. Such a star!

Also got to talk to  mikemcgarry  from Magoosh, and he was sad that he hasn't met  Bunuel  in person so far! Also got to meet Stacy Koprince from Manhattan GMAT, Andrew from Kaplan, Scott from Veritas Prep, and other folks from other test prep companies, and had a lovely chat with most of them in the post summit open space. I will definitely try and revisit the summit in the summer.

That's it! Post for any other comment/queries and I'd be happy to share any other thoughts from the summit.

Manhattan GMAT Summit debrief

souvik101990 · Answer

Sessions Overview GMAC annual summit 2014 - Session Overview Validity and IR Scores - Eileen Taleno-Miller Interpretation of IR Scores - Validity and Factors Affecting Validity Recent Validity Studies Proposed use of the GMAT test is two-fold - Admission decisions and prediction of success at business school. Significant correlation with IR score and business school GPA (with advanced correlation algorithm to remove certain biases) Validity studies in 15 programs, 15 schools and 8 countries include A) High GPA scores for business and econ majors, high quant scores for STEM candidates. IR has the potential to become a significant differentiator as IR scores are significantly varied in STEM and Humanities and SS candidates B) Demographic validity studies indicate high verbal scores in US and Canada and high quant scores in the Asia&Pac region. Again, significant scope of IR to be a lead differentiator. Employers have reported to want to use IR for recruitment, and potential employers may ask for score. (as a differentiator) New Score Report Option is Coming! gmac-annual-summit-188582-20.html#p1444869 Session - Psychometrics Update - Fanmin Guo Demographic averages of Quant, Verbal, IR and AWA scores. North America continue to dominate to have high verbal scores, and AsiaPac region tend to have higher quant scores. This leads to interesting trends in terms of differentiating yourself in a business school admission process based on your geographical origin. GMAC is committed to IR - a mild hint on more IR in the future (very mild hint

) New product discussions - GMAC collected feedback from the summit participants on recent product launches such as the IR Prep tool, the Question Pack 1, and the Exam Pack 1. It's a continuing trend to provide test takers with more productive tools to prep for the GMAT. GMAC continues to show commitment and responses to user feedback, especially by launching better products to supplement the official guides for preparation and practice. GMAT Item (Question) quality assurance - How does a GMAT question move from experimental to a real test question. (Let me know if you want to know more about it. Definitely the most fascinating bit in the summit) - more on next post. EVERY question counts in scoring (first questions right - Myth debunked Again!) NEW Announcement - Diagnostic score reporting - Score preview before cancelling. Score will still yield a "C" on the official test report.

egmat · Answer

Souvik.. did not realize that you are in US now.. BTW..this is great news.. I am sure that this will help test takers world wide...

dabral · Answer

souvik101990

Thanks for the news. I am assuming this is for the real GMAT test and not just the official tests(GMATPrep and the Exam Pack 1 extension). This will really help students who have to to take the GMAT second time to improve their score, this will also take some mystery away from what may have happened during the actual exam.

I hope they eventually give details similar to the GRE diagnostics. If anyone from GMAC is reading this, please take a look at the attached diagnostics for the GRE exam. I would love to see the same for GMAT. This will be of tremendous help for the students.

Cheers,
Dabral

mvictor · Answer

Wow, if they provided this kind of information, it would definitely help re-takers.

AliciaSierra · Answer

Awaiting for more information from Souvik. 
Can you pls provide detail information.

Regards,
Ammu

bb · Answer

I have been trying to get a hold of him for 2 days now. Good Luck! He is partying in New York :lol:

VeritasPrepBrian · Answer

Don't let me steal Souvik's thunder, but one update that he may be a bit bashful about reporting:

Souvik game a phenomenal presentation right after lunch, sharing GMAT Club data trends and mapping them against the overall trends in MBA applications and in GMAT test-taker data. Really a well-done and informative presentation, and Souvik represented GMAT Club extremely well.

I'll let Souvik share all the details, but my overall takeaway is that GMAC continues to be really motivated to ensure a fair test (they broke down their processes for filtering out questions that could be construed as culturally or gender biased) and to providing students with opportunities to achieve their highest possible score on the test (with products like the new diagnostic report on your official exam, as Souvik mentioned). A great Summit overall, and thanks to the GMAT Club team for a great presentation!

m3equals333 · Answer

Hey guys, I attended the event with Souvik on behalf of GMAT Club (in my capacity as a GMAT Club volunteer moderator)...I live a very short commute from NYC. I can provide a brief write-up soon, but I think other than the new score report, the most interesting discussion concerned GMAT marketing and also the general criteria/process that GMAC uses to vet a question (most interesting to me was how they have to periodically update the question pool to properly take into account current events and how a test taker may react; also the extent to which GMAC test writer go to ensure the test is free of any potential geographical/cultural/socioeconomic/etc bias). Also, coincidentally, having taken the GMAT only months ago, I was sent the e-mail by the GMAC to take a survey on the scoring diagnostic package (not sure what range GMAC used to send out the survey e-mails, but I took the GMAT 3 times total, however those 3 times were within 3 months, the first exam in May 2014 and last July 2014)...I guess the stars aligned as it came just as I was checking my personal e-mail on my phone and so I took the survey almost immediately, and I ended up being one of the first 500 to answer it (GMAC presenter said the 500 were filled within 10 minutes or so)...so I will get a free scoring diagnostic from my July 2014 test sometime in December

Excited to see what the preliminary/final product will look like, but to be completely honest....I am most excited/intrigued to see my personal July 2014 scoring diagnostic and how my actual performance, broken down by category, compares to my perceived performance

Just as a fyi, once live, the scoring diagnostic availability will be retroactive back to a certain date (I don't remember the date/range at the moment, but will try to get it and update this post). Souvik made a great presentation! A lot of really interesting information, especially certain relationships that can be derived from GMAT Club site traffic. A lot of great people there, including those from the top GMAT vendors and GMAC/Wiley. Very nice atmosphere and the lunch buffet was GREAT

Best, Derek Note: No info on the scoring algo was divulged, other than that GMAC has multiple algos and can (maybe even do?) switch it up as they see fit

ENGRTOMBA2018 · Answer

Hi Souvik

Thank you for the information. But I wanted to know more about the "Diagnostic score reporting" feature. When will it be implemented and will it be retroactive (and from when?)? I dont see any press releases on mba.com or gmac.com for the same.

Thanks

souvik101990 · Answer

Thanks! I had a ball interacting with all of them in the summit! Was in the US only for a few days though!

souvik101990 · Answer

You're too kind VeritasPrepBrian . Hope I get to catch up on more stories about how more books of the new Veritas series came into ideation

souvik101990 · Answer

It really was! But it was a very welcome change from the Mumbai weather to be honest!

souvik101990 · Answer

Session overview and thoughts on the summit posted!

ENGRTOMBA2018 · Answer

Souvik that was a great briefing! I am very curious to see if GMAC is going to provide a 'diagnostic report' for my gmat on october 24, 2014 as I never got any email for filling up any survey after my GMAT. I dont mind paying whatever amount for such a report as I am definitely going to retake my GMAT in a couple of months and this report will be very useful if we can get a report similar to what Dabral has posted. Is there a link for all the news postings from summit on GMAC.com or MBA.com? Has GMAC provided a timeframe for rolling out this feature?

Thanks for the information!

souvik101990 · Answer

I think you SHOULD be able to. Can't be sure till we see the official release. Also, I do not think the diagnostic report will be close to the GRE report. THe GRE and the GMAT work in very different ways. Expect to see sectional percentiles and some further breakdown. I doubt GMAC will give you question level performance data though.

bb · Answer

Do you think they will tell one how many questions they got wrong? or not really either?

souvik101990 · Answer

No. I am afraid number of questions correct vs attempted or any data like that is not going to be featured in the test report. But again, my predictions are based on the diagnostic score preview slides that we saw on the summit; there could be changes in the final product.

VeritasPrepBrian · Answer

Regarding the diagnostic report on the official exam and what it will contain...

The current installment is a pilot program for GMAC to gather feedback on what will and won't be useful, actionable information for users. The plan is to charge $24.99 (that could change, but that was the number they cited) for students to access the report, so GMAC wants to ensure that it's providing fair value for that price.

As of now, I'd argue that the most useful feature will be a question-type-by-question type adaptive score to tell you how you performed on, say, Sentence Correction vs. Critical Reasoning and Reading Comprehension. Right now you just get you overall scaled score of for example 39, but this would give you a similar score for each question type. If you were 39 on Verbal but 32 on SC, you could glean from that that you really need to emphasize SC. And even if you get a feeling right now from how the test felt, this would be useful as actual data.

They're also piloting a pacing metric, but as a lot of us pointed out if you rush on the last 4-5 questions but still finish in 75 minutes, your pace per question will be the 2:04 Q / 1:47 V that just divides 1:15 by 37 or 41 and might not be that helpful, so they're tinkering with other reporting possibilities to provide meaningful data.

All that said, they didn't rule out item-by-item correct/incorrect (and that was a question specifically asked) but my guess is that they won't do it. (not that they're secretive at all about the scoring algorithm, but they likely don't want to promote a culture of trying to dissect item responses in order to "crack" the CAT, and also without the questions there the item-by-item probably isn't all that diagnostic, so they also have a vested interest in not encouraging people to memorize questions to try to dissect them later)

But the bigger story - GMAC continues to push to be more transparent so that those who are working hard to maximize their GMAT scores have more resources and opportunities to do so. AND this tool is still in development and beta specifically so that they can determine what would be most useful to test-takers, so it's not a bad idea to keep brainstorming your wishlist publicly on GMATClub so that good ideas get out there and have the potential to be included.

kushwahanuj28 · Answer

Hi bb

Would you kindly explain what is happening . Could you summarize what is GMAC upto ?

Is GMAT in a way going to provide Diagnostic report as we get in mocks ?
 
Regards

	Magoosh increases your score, guaranteed - 15% off ALL Magoosh GMAT plans with code GMATCLUB15 + Free GMAT Club Tests.
	Manhattan Prep: $79 Bonus Score Higher on Your GMAT Exam. Guaranteed.
	Target Test Prep - Save up to $325 $200 Discount + $79 Bonus GMAT Club Tests
	e-GMAT: 60% Off on GMAT Focus Get to 735+ from any starting point
	Experts' Global - ~~$1720~~ $960 End-to-End GMAT Prep + App Support: 150+ Videos, 2000+ Questions, 15 FLTs +
	GMATWhiz - $299 ~~$499~~ Experience the course through a free trial.
	GMAT Club Tests - Free Included with every course purchase of $149+ View all deals

GMAC annual summit 2014

Prep Toolkit

Top 5 GMAT Debrief Videos

615 to 715 in 15 Days (Ria's Strategies)

More than Quant/Verbal Abilities: Speed vs. Accuracy

745 with Only Self-Prep and GMAT Club

From 555 to 765: 210-point Improvement

Perfect 805 Score Debrief by Julia

My Rewards