Interfaith dialogue is "absolutely vital", Sajid Javid told an audience of Orthodox rabbis in north London this week.
Addressing the Chief Rabbi's pre-festivals conference, the Communities Secretary said: "If we're going to live with each other, work with each other and tolerate each other, we have to understand each other.
"It needs to be about more than the Chief Rabbi talking to the Archbishop of Canterbury. It needs to be more than Jeremy Lawrence, the senior rabbi here at Finchley, talking to Imam Maulana Oussama Sahmoui at the North Finchley Mosque.
"Interfaith dialogue needs to happen at all levels of society. It needs the ordinary members of your synagogues, the congregation from the local church, the people who attend the mosques and temples and gurdwaras.
"All of us need to come together and see just how much common ground we share."
For its part, the government's New Neighbours programme had helped establish projects which broke down barriers.
"We've seen Jewish, Muslim and Christian organisations in, for example, Leeds working together to set up a café where people of all faiths and none can get to know each other. We've seen the Nottingham Liberal Synagogue partnering with a local Muslim organisation to provide hot meals to vulnerable people."
Despite interfaith initiatives, "there are still far too many people facing threats, intimidation and even outright violence simply because of who they are. And much of that hatred is directed towards Jews."
Mr Javid was "saddened but not surprised" Yad Vashem's Professor Yehuda Bauer had said he would be concerned about antisemitism if he were a British Jew.
"The Shoah began with nothing more than words. That's why it's so crucial that extremism, racism and violence is stopped in its tracks.
"Yes, we must do all we can to tackle the criminal manifestation of antisemitism and other religious bigotry. But we also have to deal with the underlying attitudes that fuel it."
Rabbis at the conference also heard plans for the Chief Rabbi's Awards, a Duke of Edinburgh Award-style scheme targeting bar/batmitzvah celebrants. Participants could engage in activities including studying modern Hebrew, volunteering in and outside the Jewish community and charitable fundraising.
The framework for the three-year programme is being rolled out by the United Synagogue's young people and young families department. Its working group included rabbis, rebbetzens and representatives from Tribe, the US youth arm.