The same thought assails me whenever we reach one of those dark anniversaries – Hitler’s seizure of power, the Nuremberg Race Laws and last weekend, Kristallnacht. How could they? What rational process led millions of otherwise practical modern Europeans to endorse the idea that a small and not remarkably obvious minority were the authors of everyone’s ills?

The thought never stops arising because the answers I give myself never quite work. Think what someone would have to believe to be a 1932 citizen of say, Stuttgart, walking near the Staatstheater, and just beyond the ballet posters announcing Swan Lake reading a banner proclaiming that “The Jews are our misfortune” and nodding along in agreement. In 1932 there were 520,000 German Jews in a population of 67 million — 0.8% or less than one in a hundred. A third of those lived in Berlin, and 70% of Jews lived in cities. German Jews were highly assimilated and one obvious reason why the Nazis commanded Jews eventually to wear the yellow star, was that it was far from obvious much of the time which of the people in the shops and streets were actually Jews. In other words, as minorities go this was a relatively low key one.

In recent weeks I’ve been caught up in a polite argument with a group of academics and others who are keen to explain the growth of populist nationalism in terms of an understandable response to immigration. In essence, they believe that white majority populations have responded to the pace of new immigration, especially by those who seem culturally and physically distinct, by embracing (mostly) far right movements. Furthermore, they go on to argue, such negative feelings may not be stigmatised as “racist”, but should rather be seen as a “legitimate” concern about the weakening of their cultural dominance.

This argument suggests that actual facts about migration and minorities are unimportant, rather it’s just the way people “feel” about changes around them (and who they blame for those changes) that count.

This logic terrifies me. The experience of the Jews suggests that assimilation doesn’t protect you any more from violent prejudice than being visibly or audibly different. Indeed it has been one of the arguments about the pre-war experience of German Jews that trying to fit in laid them open to the charge of cunning infiltration. More than that, Jews were openly and demonstrably not the people who the far right accused them of being. They didn’t have enormous hooked noses, appalling parvenu taste in clothes and lust openly after blonde Mädchen. As we know, not a few had fought for Germany in the Great War and had earned the Iron Cross. And given the ordinary German’s experience of Jews, race theory must have seemed just that — a theory undermined by everyday encounters.

But, even in peacetime, Hitler’s accession to power on the back of a minority of German voters became an official rhetoric of blaming everything — unser Unglück — on one group. It made no sense, but that lack of sense led to Jewish exclusion from public life, beginning with the banning of kosher slaughter and by 1936 banning Jews from parks and swimming pools and forbidding them the use of bicycles and typewriters. What feeling of the average German could possibly be propitiated by banning a Jew on a bike?

Jews on bikes became Jews acting against the nation and that in turn became Kristallnacht. In January 30th 1939, just ten weeks after the burning of the synagogues, Hitler spoke to the Reichstag about these Jews who, remember, made up such a small part of his new Reich. First he mocked the Western powers for protesting on behalf of German Jewry. If the West thought Jews were so wonderful, he mocked, why was it so reluctant to take them? And he finished with this: “Today I will once more be a prophet: If the international Jewish financiers in and outside Europe should succeed in plunging the nations once more into a world war, then the result will not be the Bolshevisation of the earth, and thus the victory of Jewry, but the annihilation of the Jewish race in Europe!” There is a great round of hoorays and applause.

I am not posing this thought in order to win an argument by simple recourse to the Nazis and the Jews. I am posing it because it genuinely bothers me, almost more than I can express. If in matters of race or creed or difference, you allow the validity of the irrational prejudice — they look or sound or cook or seem different or impinge in some imagined way on the life of my group, therefore I am affronted — then why wasn’t German dislike of the Jews just as valid?

David Aaronovitch is a columnist for The Times