It's more like that Jewish people were the majority of victims in the Holocaust, but plenty of other "undesirables" or political enemies were killed as well. A bit difference though is that the Nazi government made a point to demonize the Jews, push propaganda against them, and often singled Jewish people out with their policies like the armbands or the ghettos. The movement was against Jewish people specifically, but was also about getting rid of other groups of people the Nazis didn't like or thought weren't good enough.