I am white. I am privileged because I am white, if I argued otherwise I think most people on this forum would have words. I am also autistic, trans, aro, and ace. I am discriminated against for all four. Just because you have some privilege does not mean that you also do not experience discrimination, oppression or stigmatisation and there is no dissonance in that statement.
And related to your question, yes they do. But they do experience arophobia and acephobia respectively, things I have personally witnessed and the gay people have some measure of privilege for not experiencing that. Which does nothing to diminish the very real problems any of those three hypothetical people face.
Like I said originally, this whole concept isn't meant to be used as a measuring stick. It's a way to acknowledge issues and needs that various groups face and how they may inter-relate and connect to each other.
Do you think it's ok for transphobes, gay or otherwise, who dislike being called cisgender to shout down that word? It's literally the same situation. No one is asking people to self-id as alloromantic or allosexual, but the words are still useful for us to talk about people who aren't us and taking away vocabulary is a form of harassment and has been a common tactic for silencing people, as has already been mentioned.