closeicon

Beware of fishy survey methods

Researching Jews, like researching any small, geographically skewed, hard-to-reach group, is an incredibly difficult thing to do, writes Jonathan Boyd

articlemain
August 31, 2017 12:53

Let’s suppose, just for a moment, that you’re in the smoked salmon business. You’re keen to do well, of course — you want your parents to be proud — so you’re eager to calculate the size of your potential Jewish market. You commission a survey. And, after a short time, the results come in: you discover that 25 per cent of British Jews like smoked salmon. A ridiculously low estimate, I realise, but bear with me — this is just a thought exercise.

So how big is your market? Surely the calculation is simple. The 2011 Census demonstrates that Britain has a Jewish population of 265,000. Twenty five per cent of that equals 66,250 people. Done. Easy-peasy.

But hang on a minute. That figure is just for the Census of England and Wales, and you want to know the size of your UK market. So you need to refer to two other censuses — for Scotland and Northern Ireland — to find the total UK count. And that takes the total up to 271,000. Not a big difference, granted. But it does expand your market slightly, to 67,750 people.

OK. Are we done now?

Errr, no. The 2011 Census religion question was optional, so a certain percentage of Jews simply didn’t answer it. The 265,000 figure for England and Wales must be, by definition, an undercount. But by how much? There are different ways to make an assessment (and there have been several lengthy academic journal articles written about it) but most lie in the range of 7 to 15 per cent. In that case, the true population count lies somewhere between about 285,000 and 312,000. So, if we apply our 25 per cent figure now, we find that actually, between 71,250 and 78,000 British Jews like smoked salmon. Add the Scots and the Northern Irish in too, and calculate the undercount factor there as well (a different calculation for England and Wales), and we end up with a total market size of somewhere between 73,000 and 80,000.

That’s encouraging. Can you find me any more Jews?

Sure – no problem. Different types of Jews like smoked salmon. Affiliated Jews, unaffiliated Jews; in-married Jews, intermarried Jews; halachic Jews, non-halachic Jews; Jews by religion, Jews by ethnicity; members of Jewish households (who may or may not be Jews, but let’s count them anyway), very frum Jews, completely assimilated Jews, children of Jews, grandchildren of Jews, and Jews who don’t even realise they’re Jews. Let’s throw them all into the mix. How many now? Demographic estimates for the number of Jews by heritage sum to 410,000. So, in fact, your real potential market is 102,500 people.

Now we’re talking. This kosher smoked salmon business is really going to fly!

Ummm, wait a second. Did you say “kosher” smoked salmon business? Because, if that’s your business, you’re only likely to appeal to the more engaged and affiliated part of the community. And that’s much smaller. Probably no more than 225,000 people in total, including children. Once you take those factors into account, you’re talking about a market of about 56,000; without kids it’s only about 41,000.

Thought exercise over. Let’s put the smoked salmon back in the fridge. Here’s the point. We just learned that 25 per cent of the UK Jewish population could add up to anything from 41,000 to 102,500 people.

So when someone tells you that x per cent of Jews thinks a, or does b, or believes c, it’s critical to know exactly how they built their sample, which definition of “Jewish” they employed to make their assessment, who made that assessment, what tests they conducted to determine the accuracy of their results, and who checked and verified their figures. Because, to be brutally honest, this an immensely complex business.

That’s why highly credible social scientists vigorously debate whether the size of the American Jewish population is 5.7 million or 6.7 million, or whether the Jewish population of the FSU is 267,000 or 866,000. That is also why, in almost a decade of working at the Institute for Jewish Policy Research, we’ve only conducted three national surveys of Jews. It’s because researching Jews, like researching any small, geographically skewed, hard-to-reach and rather amorphously defined group, is an incredibly difficult thing to do without meticulous planning, painstaking, detailed and time-consuming statistical work on any sample created, and very careful analysis of the findings.

Actually, given that, maybe a career in the smoked salmon business instead isn’t such a bad idea after all?

Jonathan Boyd is Executive Director of the Institute for Jewish Policy Research (JPR)

August 31, 2017 12:53

Want more from the JC?

To continue reading, we just need a few details...

Want more from
the JC?

To continue reading, we just
need a few details...

Get the best news and views from across the Jewish world Get subscriber-only offers from our partners Subscribe to get access to our e-paper and archive