I've felt this way a couple of times too but in my case, I'm just aro, not ace. I've thought about it a lot and I think I've figured out what caused these feelings (for me at least). What I found helped was separating "love" into different kinds of love. I think it could be different for everybody but I personally have created the groups familial, platonic, romantic, sexual, aesthetic, celebrity, inanimate, and fandom love. What I found was happening was I was feeling platonic love (I wanted to be their friend), sexual love (they were hot), and aesthetic love (they looked nice) all at the same time but due to the combination of these types of love in common media and the fact that many of them don't have set boundaries and could overlap, I had thought I was feeling romantic love.
How you approach the situation from here is your choice, I guess. You could become closer friends but not go any further than that or pursue a relationship but make sure they know how you feel on the matter or whatever makes you comfortable!