I agree that they don't have to be straight, but they have to be good enough actors to convince me they fancy the girl. Same with straight actors really, I've seen plenty of movies/shows with straight guys (and girls) who don't for the life of them seem into the person they are supposed to be. I guess it comes down to being a decent actor in the right part.

Barney is a good example, you can't tell he's gay, though the character is mucho confused about himself you never question that he digs any kind of chick he can get his hands on. George on GA, when I've watched it the character always seemed very asexual and charisma to me (before and after he had to come out as gay) though I think that might be the way he plays the character and the writing, because I can't imagine George have sex with guys either.

About personally finding them attractive, I would care if they were gay (because zero actors are who their media persona says they are and many of them are made up every day anyway), but I know some people who would be bothered by it.