Summarizing 20 years of US News & World Report Rankings

Question

Tonight, I scraped the internet looking for legacy rankings for on-campus MBA rankings for US schools. Frankly, I was surprised I was able to find as much as I did ... as quickly as I did. The fine folks at USNWR don't make it easy, that's for sure!

As with any analysis of small data sets ... caveat emptor.   The biggest assumption made is the volatility in the historic rankings of these schools over the past 20 years will generally reflect the volatility over the coming 20 years.

Results: 
1) For each school with a ranking I could find (some years, I only had 20 schools, some 15, some 40+ and one year, only 10), I averaged the rankings given to that school.
2) This generated a bit of bias towards schools who were usually listed in the Top X schools, but weren't in the Top X in a year I only happened to find a few schools. For example, in 2006, Yale was not a top 15 program ... so, by eliminating their score (which was AT LEAST a 16, and maybe lower, it artificially increased their average score --- so --- to address this, and using Yale as an example, in all other years, Yale's average ranking was 11.5, but they weren't in the top 15 ... so ... I'd add 35% to their average (and round), which means that for 2006, Yale was ranked at 16 (11.5 x 135%, rounds to 16), and I'd keep doing this for schools (in decending order until I had at least 20 schools in a year). I also had to do this for the following schools in the following years (and the rankings given to each in those years):

2002 (top 10 only) -- Berkeley (11), Yale (16), NYU (16), Virginia (17), UCLA (18), Cornell (21), Texas (23), Carnegie Mellon (23), UNC (26), Emory (27), and USC (29)
2003 (top 16 only) -- Texas (23), Carnegie Mellon (23), UNC (26), Emory (27), and USC (29)
2006 (top 15 only) -- Yale (16), Carnegie Mellon (23), UNC (26), Emory (27), and USC (29)
2007 (top 26) -- USC (Marshall) wasn't in the top 26 that year, but is (otherwise) in the top 20 here ... so, I put in a 29 for it for this year
2008 -- Emory (Goizueta) got a 27 this year for the same reason USC (Marshall) got a 29 the prior year
2010 -- Emory (Goizueta) got a 27 and USC (Marshall) got 27 / 29, respectively
2011 -- UNC (Kenan-Flagler), USC (Marshall) and Emory (Goizueta) got 26, 27 and 29 ... same reasons ... 
2012 -- Emory (Goizueta) got a 27 and USC (Marshall) got 27 / 29, respectively

..........................

Now, with the blanks filled in as reasonably as I was able given my constraints, the ranks were averaged, a slope was determined as was a standard deviation.

Once a school is at the top of a ranking, it's pretty hard for these schools to actually have a positive slope to their scores (i.e., the only place for them to go is down!) ... interestingly, the only schools in the top 10 for the last two decades with a positive slope was Chicago (Booth) at #6 and Duke (Fuqua) at #10. Other schools in the top 20 with positive slopes were Yale, NYU (Stern), UCLA (Anderson), Cornell (Johnson), Carnegie Mellon (Tepper), Texas (McCombs) and UNC (Kenan-Flagler).

..........................

This first list is in descending order of the school's average rank for the past 20 years -- but -- the #s next to each one reflect the most recent ten years / most distant ten years to give a rough idea for how the schools are moving up / down / sideways:

Harvard - 1.9 / 1.3 ... -0.6
Standord GSB - 1.8 / 1.6 ... -0.2
Penn (Wharton) - 2.3 / 3.2 ... +0.9 ... nearly as impressive as Chicago (Booth)'s increase, but not quite ...
MIT (Sloan) - 4.3 / 4.2 ... -0.1
Northwestern (Kellogg) - 4.9 / 4.1 ... -0.8
Chicago (Booth) - 3.5 / 6.2 ... +2.7 ... a very (!) impressive increase decade over decade, especially given they started off at such a great rank to begin with!
Berkeley (Haas) - 6.9 / 7.9 ... +1.0
Columbia - 8.3 / 7.4 ... -0.9
Dartmouth (Tuck) - 9.3 / 8.8 ... -0.5
Duke (Fuqua) - 11.9 / 10.2 ... -1.7 ... second biggest drop, decade over decade ...
Michigan (Ross) - 11.5 / 11.3 ... -0.2
Yale SOM - 10.5 / 13.4 ... +2.9 ... second biggest riser in the top 20
NYU (Stern) - 11.9 / 12.3 ... +0.4
Virginia (Darden) - 12.0 / 13.4 ... +1.4
UCLA (Anderson) - 15.2 / 13.1 ... -2.1 ... biggest drop decade over decade ...
Cornell (Johnson) - 15.8 / 16.2 ... +0.4
Carnegie Mellon (Tepper) - 18.3 / 18.4 ... +0.1
Texas (McCombs) - 16.9 / 19.9 ... +3.0 .. biggest mover in the top 20
UNC (Kenan-Flagler) - 19.4 / 21.7 ... +2.3
Emory (Goizueta) - 21.5 / 24.0 ... +2.5

.......................................

The rankings below are for the top 20 schools' ... and is based on their average over the past 20 years. The numbers in the parenthesis reflect 2 standard deviations on either side of the average, in an effort to give everyone a rough idea for where schools might reasonably be considered to be in twenty years' time  (if past = prologue, that is) . I didn't fiddle with anything in the list immediately below to reflect the positive / negative slopes. The "two decade top 20" is:

Harvard -- 1.6 (range of 1.0 to 4.0) ... note: for schools at the very top, reducing their average score two standard deviations from their average would actually give them a 'negative rank', and since that's not possible, I made it 1.0
Stanford GSB -- 1.7 (range of 1.0 to 3.5) ... meaning, Stanford had less deviation in their year to year rankings than Harvard, hence their better 'worst case' ranking of 3.5 vs. 4.0
Penn (Wharton) -- 2.8 (range of 1.0 to 4.8)
MIT (Sloan) -- 4.3 (range of 2.9 to 5.7) ... smallest standard deviation of the top programs
Northwestern (Kellogg) -- 4.5 (range of 2.5 to 6.5)
Chicago (Booth) -- 4.9 (range of 1.0 to 8.9) ... biggest standard deviation of the top programs ... and when combined with their positive slope, would give the the best argument of schools in this group to turn the top three into the top four ...
Berkeley (Haas) -- 7.4 (range of 5.2 to 9.6)
Columbia -- 7.9 (range 5.5 to 10.3) 
Dartmouth (Tuck) -- 9.1 (range 6.3 to 11.9)
Duke (Fuqua) -- 11.1 (range 6.8 to 15.5) ... best case nearly matches Dartmouth, but lower starting average and bigger volatility in ranking keeps them from being in the same tier
Michigan (Ross) -- 11.4 (range 8.2 to 14.6) ... a lower risk school (if rankings is all that matters) than Duke, Yale, NYU and Virginia ...
Yale SOM -- 12.0 (range 7.2 to 16.8)
NYU (Stern) -- 12.1 (range 7.3 to 16.9)
Virginia (Darden) -- 12.7 (range 9.1 to 16.3)
UCLA (Anderson) -- 14.2 (range 10.0 to 18.4)
Cornell (Johnson) -- 16.0 (range 13.0 to 19.0)
Carnegie Mellon (Tepper) -- 18.4 (range 13.8 to 23.0)
Texas (McCombs) -- 18.4 (range 13.4 to 23.4)
UNC (Kenan-Flagler) -- 20.6 (range 14.8 to 26.4)
Emory (Goizueta) -- 22.8 (range 16.0 to 29.2)

.....................

This means that a reasonable argument could be made that Texas (McCombs), UNC (Kenan-Flagler) and Emory (Goizueta) are the most likely to find themselves OUTSIDE the top 25 at some point.

It also means that while Chicago (Booth), Berkeley (Haas), Columbia, Dartmouth (Tuck) and Duke (Fuqua) are all ahead of Michigan (Ross) today, there does exist a possibility Michigan (Ross) could find themselves ranked in front of all of those schools at some point in the next two decades.

+++++++++++++++++++++++

This list makes an effort to do two things -- tier the schools based on averages AND slopes ... schools in a tier are listed alphabetically:

Tier 1 ... have a hard time seeing any of these ever being out of the top five
Harvard, Penn (Wharton) and Stanford

Tier 2 ... have a hard time seeing any of these ever being out of the top ten
Chicago (Booth), MIT and Northwestern (Kellog)

Tier 3 ... have a hard time seeing any of these ever GET INTO the top five
Berkeley (Haas) and Columbia

Tier 4 ... have a hard time seeing this one too far on either side of 10th for very long
Dartmouth (Tuck)

Tier 5 ... have an easy time seeing these crack the top 10 from time to time and rarely, if ever, getting out of the top 15 as a worst case scenario
Duke (Fuqua), Michigan (Ross), NYU (Stern), Virginia (Darden) and Yale SOM

Tier 6 ... might crack the top 15 from time to time, but are pretty comfortably in the top 20 as a worst case scenario
Cornell (Johnson) and UCLA (Anderson)

Tier 7 ... while it'd be really tough, one or two might crack the top 15, but each one of them has a chance to fall out of the top 25
Carnegie Mellon (Tepper), Emory (Goizueta), Texas (McCombs) and UNC (Kenan-Flagler)

{}{}{}{}{}{}{}{}{}{}

SUMMARY: If where your school is ranked is important to you, uber alles, you might want to take this into consideration when deciding whether or not it's worth it to save a couple of bucks to attend one school ... or to reapply next year ... or ... or ... or ... etc ...
 
Good luck to everyone!

MBAPrepCoach · Answer

This is some really great analysis and I like your conclusions and agree with them. I think the only addition I would say is that it's hard for me to see Booth out of the top five.

Posted from my mobile device

Regenerate · Answer

Because of trend, absolutely.

Because of prior positioning, I think they're good where they are.

In five years' time, a different answer could very easily be reached (especially if HBS doesn't bounce back).

fatmike · Answer

Awesome work, some good stuff! I definitely think it should be HS/WB these days, but maybe I am a little bias...

Posted from my mobile device

AndrewN · Answer

Thank you,  Regenerate , for sharing all that detailed information. If you do not mind my asking, why did you start the project in the first place? Does business school lie in your own future? In any case, I can see you possess a fondness for mathematics.

Right back at you: good luck in your endeavors, whatever they may be.

- Andrew

bb · Answer

Regenerate - are you an admissions consultant or a writer or an applicant or?

Thanks for posting. I agree, yes, there are some interesting trends you can observe when charting the trends but we have the last 13 years, not 20.... though as a few people mentioned that all these TIES are really annoying when there are 6 or 7 spots really in the Top20

Regenerate · Answer

bb ... I'm just a guy considering whether or not an MBA makes sense for me at this stage of my career. I enjoy analysis and research, and communicating what I find to others in hopes they find it helpful. If there is anything I can do for you, don't hesitate to let me know.

Thanks! And, yes, I'd seen your chart. I, like others, sometime digest data in different ways; I thought I'd take a stab at constructing it a bit differently and delivering different perspectives on the same set of information built with the same assumptions. I like your chart.

bb · Answer

Awesome!!! Forgive me for asking you a direct question. We’ve had people repost some cool posts and then a wave of spam comes after that so I was wondering if I should be bracing for that

Glad that is not the case. Thrilled to have you and thank you for the analysis. Very interesting way of thinking about it. I really like your summary of the long term ranking stickiness. Welcome to GMAT Club! (Lowering the shotgun)

Posted from my mobile device

Regenerate · Answer

Thanks Andrew. Yes, I'm considering applying for an MBA. I really enjoy disassembling complex inputs in an effort to extract actionable information, and the communicate that to others as they pursue their own goals.

Facts are good things. Facts in context are invaluable.

Regenerate · Answer

Update: If you want to consider volatility as a proxy for uncertainty / instability, this list below might be of interest.

Standard Deviation / Average Score

...............

Harvard -- 1.2 / 1.6 = 0.75
Stanford GSB -- 0.9 / 1.7 = 0.53
Penn (Wharton) -- 1.0 / 2.8 = 0.36
MIT (Sloan) -- 0.7 / 4.3 = 0.16 
Northwestern (Kellogg) -- 1.0 / 4.5 = 0.22 
Chicago (Booth) -- 2.0 / 4.9 = 0.41
Berkeley (Haas) -- 1.1 / 7.4 = 0.15
Columbia -- 1.2 / 7.9 = 0.15
Dartmouth (Tuck) -- 1.4 / 9.1 = 0.15
Duke (Fuqua) -- 2.2 / 11.1 = 0.20
Michigan (Ross) -- 1.6 / 11.4 = 0.14
Yale SOM -- 2.4 / 12.0 = 0.20
NYU (Stern) -- 2.4 / 12.1 = 0.20
Virginia (Darden) -- 1.8 / 12.7 = 0.14
UCLA (Anderson) -- 2.1 / 14.2 = 0.15
Cornell (Johnson) -- 1.5 / 16.0 = 0.09
Carnegie Mellon (Tepper) -- 2.3 / 18.4 = 0.13
Texas (McCombs) -- 2.5 / 18.4 = 0.14
UNC (Kenan-Flagler) -- 2.9 / 20.6 = 0.14
Emory (Goizueta) -- 3.4 / 22.8 = 0.15

...........

The lesson ... like a good growth stock, top MBA programs have high (pseudo) betas. MIT is a notable outlier.

By comparison, Cornell looks like a preferred stock or maybe a convertible bond.

Duke, Yale and NYU are notably more volatile than other similarly respected schools ... indicating an increased risk they fall and damage your expensive diploma ... and ... an increased opportunity your degree from one of those schools might go up without you having to do anything!

Posted from my mobile device

Regenerate · Answer

It occurred to me last night as I was thinking about rankings that it seems like every year they change the way they calculate the ranks.

I have no way to go back to standardize the base formula across time.

While I get that the world changes and the formula needs a tweak every now and then, it always felt to me like the change occurred just so they could have something to talk about. And that always bugged me.

So, take everything above with an even bigger grain of salt.

Posted from my mobile device

bigge2win · Answer

Seems like you didn’t come across the source below that has the historical data going back to 1990 for the current Top 20 “ish” schools:
https://www.mba50.com/a-history-of-the-u ... 1990-2013/

You’ll find a number of sources if you specifically google “us news mba rankings historical”

Definitely shouldn’t be filling in historical data if you don’t have any just to “extrapolate” a full dataset. As you’ll see in the earlier years of the source I sent, CBS and Haas flipped places with Ross and Fuqua over the years. Also, it further shows the legacy perception of why Booth won’t be H/S/W even though modern rankings have them in the Top 3, unless Booth keeps it up for maybe 1-2 decades.

Posted from my mobile device

bigge2win · Answer

Also, a few other thoughts since I was able to read through all your posts in this thread.

1) On your first post under the section with the ranges of standard deviations, the data doesn’t actually support stuff you’ve mentioned that could be reasonably argued, like certain schools that are likely to fall outside the Top 25. For example, Tepper has the same average (18.4) and lower-bound range (23) as McCombs, but you didn’t mention them as likely to fall out of the Top 25. Also, the comment about Ross where you grouped Duke with Booth, Tuck, and CBS seems like you didn’t come to the proper conclusions where it’s Fuqua and Ross together since the data is showing Fuqua as equivalent to Ross and not the other 3 schools. There seems to be some lack of consistency in how you’re interpreting the data to even come up with your arguments.

2) On your recent post about dividing standard deviations with averages to get volatility, that seems like an apples to oranges calculation. Why wouldn’t you divide standard deviation by the difference of the range of highest and lowest rank to get volatility instead since that’s more accurately apples to apples? Dividing SD to average will yield noise further up the rankings since you’ll always divide by a smaller number. It’s a skewed metric.

3) less about your methodology, but more of a comment about the schools. USC has been climbing the rankings and shows some stickiness in the Top 20-25, so they’d be good to observe in the coming years.

Posted from my mobile device

Regenerate · Answer

bigge2win ... thanks for the feedback ... without replying line item by line item ... I didn't have the data you provided; thank you. I did the best I had with what I had; not extrapolating in years with missing data would have yielded a whole other sort of error. I mentioned that above.

If I left Tepper (or any other school) out of an obvious reference, it was more of a proofreading error than an analytical error, or worse, a desire to create confusion.

Regarding the bunching of Ross / Fuqua / Darden / NYU / Yale, they seemed to me like they fit together as a messy group than trying to parse the admittedly imperfect analysis even more finely. I'd much much much rather be generally right than precisely wrong.

I believe your volatility metric may have value. I'd want to model it before having an opinion; I appreciate you sharing it. That doesn't mean the one used does not have value, though. You might notice that after the first half dozen schools, all the rest bunch into two groups ... those at 0.13-0.15 and those at 0.20 ... except one (Cornell) that's a very boring 0.09.

Regarding USC and their stickiness ... I have only a limited opinion. We will see if it holds up over the next few years. I expect to have a more fully formed opinion by the time the class starting this fall graduates.

Again, thank you for the feedback. If I have a chance to do this again, I'll keep it all in mind.

Posted from my mobile device

bigge2win · Answer

Hmm, wondering how you’d get an error if certain schools don’t have a ranking. Are you just running a formula across values even if they’re blank? You could create formulas to dynamically adjust based on whether your dataset is populated or has a blank value, which won’t give you errors.

Also, it looks like my Ross and Fuqua comment didn’t make sense to you. It wasn’t about them being bunch with their peers (Darden, Stern, SOM). It was regarding the following point you made that I’m pasting below:
“ It also means that while Chicago (Booth), Berkeley (Haas), Columbia, Dartmouth (Tuck) and Duke (Fuqua) are all ahead of Michigan (Ross) today, there does exist a possibility Michigan (Ross) could find themselves ranked in front of all of those schools at some point in the next two decades.”

Not sure why you’d put Fuqua with those other 4 unless it was simply because Fuqua was above Ross. But Fuqua was only 0.3 above Ross as opposed to being 2-6 behind Booth/Tuck/Haas/CBS. So your observation seemed a bit nonsensical or not as valid.

Curious to how additional analysis would look like, if/when you run that.

Regenerate · Answer

The error was explained above.

Essentially, if (say) School AAA is usually ranked between 9-15, but in a particular year, a year I only had the top 20 ... and School AAA isn't anywhere to be found (meaning it's at least #21 and possibly lower) ... how should that be treated?

Do we calculate the average based only on the actual ranks we have, or do we add in a rank for the missing year that's AT LEAST a 21?

Leaving it out of the average artificially lowers the average vs what we know it should be from the inference that the missing year resulted in an outlier (low) ranking ... which also would have a big impact on their standard deviation.

Posted from my mobile device

bigge2win · Answer

I see what you mean. I guess it’s a difference of opinion on the approach. Extrapolating volatile/unpredictable data adds noise, so it’s a tricky balance to make. The approach with missing data goes three ways - just use data you have and calculate it so that you’re only averaging the years available for a school (which doesn’t artificially make the rank look better since you have a different denominator for those schools); you exclude the years with incomplete data for all schools; or do exactly as you did which is manufacture data based on some judgment on the assumptions to fill in missing data.

To my point with not filling in the data, say you only have 8 years for Emory and not 10 years for a set of 10 years. If it hovers around 22-25 and you generate rankings that get it around 27-30 for the missing years, then you’ve skewed the data based on that assumption. If you determine the average for only those 8 years, then it’s more accurate.

It’s all somewhat of a moot point now because you have the data back to 1990 in the link I provided.

Summarizing 20 years of US News & World Report Rankings

Prep Toolkit

Top 5 GMAT Debrief Videos

615 to 715 in 15 Days (Ria's Strategies)

More than Quant/Verbal Abilities: Speed vs. Accuracy

745 with Only Self-Prep and GMAT Club

From 555 to 765: 210-point Improvement

Perfect 805 Score Debrief by Julia

My Rewards