D33TWe can't use the presence of "I" to justify a possessive ("my"). The two really have nothing to do with each other. I could say "I want you to invite me" or "I want you to invite my cousin." The issue is whether the pronoun is serving as the object of a verb (me) or as a modifier (my). In this sentence, no one is objecting to ME. They're objecting to the idea that I will go to the convention. So they object to my attendance, or in this case, "my going."
This is one of those usages that is formally correct but does not reflect how most of us speak in daily life. In real life, most people would probably say "Do you mind me tagging along tonight?" Or "I'm bothered by Darren taking charge," but technically, we should say "MY TAGGING" and "DARREN'S TAKING." Certainly, some of us speak this way, especially if we are trained in SC, but this may sound weird even to many fluent and/or native speakers of English.