I've been a journalist for more than three decades, so I'm not often taken by surprise by other scribblers. But, during the flurry of media interest in our film Things We Won't Say About Race That Are True, a pugnacious radio interviewer bowled me a bouncer. If we should be able to say whatever we like about other people, he asked, should he be able to say "I really don't like black people" out loud?

There was a diplomatic answer, of course, something about every group containing some likeable and not-so-likeable folk. But he caught me on the hop, and I did the unthinkable - I answered honestly. Well, I said, I'd prefer it if you did say that - if that's what you thought. Then at least I'd know what kind of person I was talking to; and I'd be able to decide whether I liked you enough to try to persuade you think differently.

I think that this moment will resonate with Jews. Black Britons and Jews share much history. When my parents came to England from the Caribbean, for many years the only white folks my family were able to mix with were the Irish, who couldn't afford to move out of the street when we moved in; and the Jews, who didn't see any need to.

Many of our most passionate and effective advocates and supporters of race equality have been Jewish, and not just in their own interest. So I want British Jews to understand why I think it is so important to have an open debate about race and religion at a time when these issues have become such dangerous flashpoints in our society. Even if that means that facts we find difficult are brought into the open.

Jews know better than most the cost of allowing silent, unspoken prejudices to fester unchallenged by the disinfectant effect of rational debate and scientific inquiry.

We don’t bar Jews we just prefer not to take them

In March 1960, the Jewish Chronicle published a series of articles based on an investigation into what became known as "golf-club antisemitism". Margaret Thatcher, then the MP for Finchley, was quoted by the distinguished journalist Anthony Howard in 1963, as remarking that antisemitism arose in the most unlikely of places: "It's the golf club, you know. That's where it all started and where the whole thing has come from".

The JC uncovered unashamedly antisemitic attitudes at the heart of the golfing establishment. One club secretary claimed that, "we would not say we bar Jews. We just prefer not to take them", while another noted, "there is no discrimination here, but the committee never elects Jews".

This isn't about the right to play golf, of course. A century ago, the world's most distinguished seat of learning, Harvard, maintained an unofficial quota for Jews. A fair admissions system would have meant that freshman classes were dominated by brainy Jewish students. Nobody spoke about it publicly, but everyone knew it existed.

Today, we think we'd never tolerate such bigotry. There is no longer a Jewish quota. But new minority groups face the same challenge. Research shows if you are a high achieving Asian American with top grades, it will be more difficult for you to enter some of the top US universities - in some cases you are three times as likely to be rejected. We do know that discrimination still exists. But all too often it is driven by quiet, unspoken understandings.The distinguished lawyer Anthony Julius told the Guardian in 2010 that he "was subject to formal and informal 'quotas': his public school, the City of London school for boys, had a limited number of places available for Jewish children, and when he applied to law firms he was quietly advised some larger companies did not take Jews."

Today, minority Britons of all backgrounds understand that bigotry does not wear its colours openly. In the black communities we often talk of our equivalent to golf-club antisemitism, what some call, "smile-on-the-face" racism.

Few of us are paranoid about this, and we go about our daily business with the assumption that our colleagues will treat us fairly.

Yet I think that most minority parents would be fibbing if they didn't admit that we teach our children to be on the alert for the signals of rejection. That's because we know that hypocrisy is bigotry's best friend; and that silence is the enemy of justice.

That is why I took the decision to publish data about how different ethnic groups, including Jews, are faring in Britain today. I know some people will regard this as divisive. Others will say that it is dangerous to be talking about these issues at a time when all over Europe, anti-immigrant parties are peddling myths about minority groups. But my view is that this is exactly the moment to have this debate.

For many years, I believed, as did many others, that the best and safest way to protect minorities was to stress that we were just like everyone else, and to prevent people from saying anything that would lead to any one group being picked out or stigmatised.

But the answer to others' irrational prejudice should never be silence; it should be fact. There is no reason why the antisemitic conspiracy theorists should be the only people allowed to speak of Jewish success. Yes, it is true that though Jews constitute fewer than one in every 200 Brits, among Britain's billionaires' club, one in five is Jewish. Yes, it's true that Jewish leaders dominate some of our biggest and best-performing businesses, banks and property companies. It's manifest that a disproportionate number of our greatest artists, entertainers and film and TV directors, are Jewish. And yes, it's true Jewish students outperform the average at every stage of the educational process.

As the MP Luciana Berger has wisely pointed out, we should not forget there are pockets of poverty even in such a successful community. But it does mean that the answer to the conspiracy theorists should not be a hushed, nervous silence. It should be a proud declaration that British Jews are people of talent, diligence and ability, who have earned success in a free society and have made a stellar contribution to our prosperity and culture.

Frankness also means being clear about the source of contemporary antisemitism. Whether or not it's true that there is a such a rise, the evidence suggests that the source of much of the hostility is not the old far right, but the wilder fringes of the far left and hardline Muslim sects, neither of whom makes the distinction between the Jewish state and the state of being Jewish.

But unless we can name the problem, how can we ever hope to tackle antisemitism with conviction and success? But there is another, deeper set of lessons we should take from Jewish success.

First, there is no evidence it is in some way biologically determined. It is far more likely that Jewish predominance in some fields has more to do with a culture of diligence and an attachment to intellectual endeavour, combined with a healthy level of social and economic insecurity. If you fear that you may one day have to flee persecution, you know that the only thing that cannot be stolen from you are your skills and education.

Second, let's admit that the bonds of trust and loyalty are not just sentimental, but material. It is widely - but slyly - implied that much Jewish success is down to the supposed preference of Jewish business people to deal with each other and to give advantages to their co-religionists. Leaving aside whether such a preference should be regarded as in some way morally wrong in a free society, there is a rational explanation for such preferences: you may have no choice.

I learned this early in life. In the 1950s, when only slum landlords would accommodate black families, and banks and building societies would not lend to people like my parents, there was only one way to put a roof over your family's head: for immigrants to club together and share the burden of saving for a deposit. That's how my parents bought their first home. But such a system only works if you feel that you can trust everyone else in the club - and the bonds of village, church and exclusion from the wider society provided the glue that kept everyone honest.

The community from which I come is in many ways like the Jewish community. We are by and large religious, hard-working and capable. We are traditionally great entertainers and actors. But we are also very different. Unlike Jews, we are in showbusiness, but puzzlingly not agents or managers. We are spectacularly unsuccessful in the areas where Jews tend to shine. Some of that is down to prejudice. But Jews faced that, too. So the wisest people in my community have long said that we need to learn from our neighbours how they overcame that prejudice.

Thirty years ago, one of the most principled politicians I ever knew, told me and a group of other young equality warriors that we needed to study the habits of success from the Jewish community, and emulate them if we could. His name was Bernie Grant, the former MP for Tottenham, and militant anti-Zionist. But, unlike many of today's anti-Zionists, he was no antisemite. He was an admirer of Jewish traditions and especially of the Jewish enthusiasm for engagement in public life. If he were here today, he would question whether what I am doing is wise, and ask whether speaking openly about racial issues may give comfort to real racists and antisemites. But he would also say that minority groups who either have given up on politics, or imagine that the right way to improve their lot is to promote extremist ideologies, should get real. And I know he would be on the side of plain speaking. Would that today's political leaders - and the rest of us - were as courageous as he was.