You CANNOT be "anti Semitic" to just one set of people because the Jews aren't the only Semitic people on this planet. You can be anti African/Indian/Chinese and such and each of them would be identifiable to one people who basically look the same but semitic is a whole lot of different looking people, not just one...Now you know.

https://en.wikipedia.org/wiki/Semitic_people