Are you meeting them in person? If so, that's actually quite unusual. Most of them, you never meet and only have them on skype at best, but most likely phone or even email. However, if you are doing these in person, I feel these meetings are more valuable than phone or email since you exchange more non-verbal communication and if the consultant is good, they can pick up a lot.
There are a few things at play potentially: Most consultants actually have busy day jobs and some of them are super impressive individuals - truly the best of the best. Many of the consulting companies are able to tap these folks in at $100-200 per hour. Even for the HBS MBB types, it is an attractive source of income that allows them to pick who they want to work with and provides a healthy distraction. At the same time, their primary job is their day job and they have certain limitations and factors they have to balance.
At the same time feedback like yours can either indicate an over-booked individual who is really not in charge of their own time (a controlling boss who changes meetings, etc). It also can mean a disorganized person who may be brilliant but they really do not have it together.
It is frustrating, I am sure and you are paying an arm and a leg, I am sure, and you are struggling to give them respect they deserve due to constant reschedule. That makes a somewhat one-sided relationship. It is not a red flag unless they are also late on their part, assignments, etc. However, you have to kind of look at the big picture - is this going to bother you to the point that you are not going to trust the consultant you are paying? Or potentially it is going to annoy you even more as deadlines and stress come in? You have a somewhat advantage by meeting in person - would it be preferred to you if you were doing it on the phone or skype but at an exact time? now that you have established some familiarity and report, you can switch to phone fairly easily.