I am a PhD candidate in business admin (going into my 4th year) married with two kids. It is possible. But you need a very, very supportive wife. You should think carefully about what you two are willing to put up with and what you won't during your 4-5 years below the poverty line. The first two years are brutal, and then it gets harder...
As for quality of life, kids won't know the difference depending on how old they are (over 10, and they will def. know they are poor due to social comparison). But remember there is a nice pay off at the end!
A critical part of this is faculty expectations in terms of time, participation in citizenry, and whether you can work from home or not ... but whatever school(s) you want to get into, you must talk to as many people as possible to determine the level of face time and interaction that is expected. Most faculty understand that if you have a family, you probably won't be around as much. But, not all faculty see it that way. Know what the culture is and the expectations (for instance, going to a school where the program director has a family, or your advisor has a family will help... when all the faculty are single or divorced with no kids, that might be a problem).
My program understands the demands of family and are cool with it. It ALL depends on where you go and the support of your wife IMHO.
best of luck.