Hi anumchandani,
Highly competitive Business Schools (such as Harvard) only interview applicants who the Admissions Officers are interested in, so your GMAT score was likely not a 'detriment' to your application. Your focus now should be on preparing properly for that interview and 'nailing' it. As such, you might want to consult with an Admissions Expert on how best to present yourself in that situation (along with learning the type of questions that you'll likely face and how you should answer those questions).
There might still be some benefit in retesting - especially if you could make a significant improvement in your score.
1) What were your Quant and Verbal Scaled Scores for this 640?
2) When is your HBS interview?
GMAT assassins aren't born, they're made,
Rich