I would go to Kellogg.
You're talking about schools on the opposite ends of the spectrum (relatively speaking) in terms of student culture and cohesiveness -- with Kellogg being one of the most tight knit and friendly, and Columbia being the most cold and individualistic of all the top 8 schools.
I'll probably get in trouble by some angry CBS students/alums for saying this, but from the many clients over the years who have visited the school and given me feedback on their visits, there tends to be a common refrain: that it was the least friendly of all the schools.
Again, it's all relative. It's not like CBS is full of back-stabbing trolls, but given the environment and the lack of a strong student culture compared to other top schools, to be blunt the school has some catching up to do.
Columbia can take a cue from Booth. For the longest time, Booth had a lukewarm to bad rep for having a socially awkward student body that was devoid of cohesiveness, all located in a crappy part of town (sort of like Columbia not being located in the best part of NY either, although the area has improved a lot since the 1980s). But UChicago really made a concerted effort to change that. It's not about endowments or shiny new buildings (although that helps) or moving to a new part of town, but about an administration, admissions staff and student body collectively working together to make a real change. And that really starts with the administration. In a way, Booth took its cue from Wharton, which transformed itself similarly in the late 1980s when it had that reputation for being a hypercompetitive a**holish place full of Trump wannabes in a crappy part of town in an already rough city (Philly isn't as bad as people make it out to be, but some folks would have you believe it's like downtown Detroit) -- to a school that has a very cohesive, active and friendly student body (the MBA program at least!). Like Wharton, Booth never turned its back on its heritage or reputation (finance), but sought to expand and build upon it. Booth to this day may still be finance driven, but it's becoming more and more diverse and sort of is Wharton Jr. (and I mean that in the best way possible).
Even though Booth still gets saddled with that reputation for the "no fun geeky" school, in the last few years (esp. last 2 years), I've had clients who tended to be pleasantly surprised about Booth - that it's more cohesive than they expected, the people were more friendly and down-to-earth and sociable than they expected, etc.
Anyhow, back to Columbia vs. Kellogg.
Keep in mind that unless you're loaded, as you know NYC is a VERY expensive place to have fun. It's a city that is a lot more fun when you're making money than as a broke-ass student. Depending on the size of your fellowship, your total cost will still be significantly higher at Columbia than Kellogg simply because the cost-of-living difference between NYC and Evanston is quite large - and that's not factoring in the "fun" stuff (eating out, social events, etc.).
Of course, if you've got a lot of money, cost won't be an issue. But if it is, being in NYC and living like a student isn't as fun as one may think, especially compared to being a student in a college town where you'll likely forge stronger and broader friendships with your classmates since they don't scatter all over the city after class.
In other words, what is more important to you -- networking/having a great time within the b-school (irrespective of its home city), or living in New York City?
The fact that you've been able to get in touch with Kellogg alumni and have had no luck with the Hermes Society isn't a rare coincidence.
Columbia is a great school when it comes to academics (some amazing profs) and recruiting (especially finance), but when it comes to student culture, it has some work to do compared to its peers like Kellogg, Booth, Sloan and Tuck. Again, it's not bad, but it's not as strong as the others when it comes to student culture and cohesiveness. It has to stop touting the "New York Advantage" because it can easily become an excuse for complacency and not doing anything to improve the actual student culture and environment. I mean, NYU Stern could also claim "the New York advantage" in an even more compelling way since it's actually in an amazing part of the city, but instead it has managed to create a culture that in spite of being in a gigantic city - is actually quite laid back, down-to-earth, friendly and tight knit.
Again, Columbia can be a great choice if you're really focused on finance and you need or strongly prefer to remain in NYC for personal reasons (i.e. spouse/partner works and you don't want to live separately). But if either of these don't really apply to you, then you're probably better off going to another peer school like Kellogg if given the choice.
In sum, Kellogg will more likely give you the quintessential college town experience (in the best way possible) -- and while you can't always go and eat at Nobu, you will likely have more money in your pocket to go on weekend road trips (aka "benders" and have your own real life versions of "The Hangover" film) with your fellow Kelloggians to NYC.