I think I only really find people attractive when I'm already horny. I never get distracted because someone's attractive, but I will find certain people more attractive if I'm actively looking for a sexual partner?
Of course this is also tied up with the fact that I'm pretty sure I have some level of prosopagnosia (faceblindness) , and I have to have known someone for a while before their face sticks in my mind well enough that I'm even able to find them attractive. And if by this point I've discovered I don't like them as a person, I'm not going to find them attractive.
So I only find people attractive when I'm horny, I've known them for a while, and we're reasonably friendly and get along and even then I don't find most people attractive, or find anyone distractingly attractive.