Is Homophobia really the wrong term?

Alrighty. So I spend far too much time online looking at news articles and such regarding LGBT people. I found that often(men), claim not to be homophobic because they hate gay people, they are not scared of gay people. Okay so that may be true. But why do these men hate gay people? Repressed homosexual urges? Fear of own emmasculinization? Myths about sexual assault by gay men? All these things stem from fear. Hatred often stems from fear and therefore the term homophobia is used accurately and appropriately.