In short, yes.
All you can do is put in a strong application and hope for the best. Don't be apologetic about where you work - but you're going to have to hope that the adcom will overlook your industry enough to admit you if he/she feels you're a strong candidate.
But yes, it matters - as it would in any other context - interviewing for jobs outside your industry, in a social setting and so forth. Some people may not care as much, whereas some people will care a lot. But few if any will see your industry in a positive light.
Adcoms are people - they have views and biases like anyone. Some have strong personal opinions on certain things, others do not, and all of that affects their view of you -- when evaluating human beings there's no such thing as "objectivity".
Follow me on Facebook