Hello,
Letters of recommendation from alumni help less than you might think. All things being equal, sure, having a letter come from someone who graduated from your target school is a minor plus. But it matters far, far more how well your recommendation writer knows you, how well he answers the questions, how many specific examples he can provide about times when you demonstrated leadership or teamwork abilities, etc.
If you're choosing between a recommender who knows you very well (usually your direct supervisor knows you best) and someone who knows you fairly well and has graduated from your target school, I would always choose the person who knows you better. The ONLY time I would choose the alumnus is if he's a very significant donor to the school. Otherwise, the admissions office is not really impressed by the fact that the person attended that school.
Hope that helps!
Scott