The founder of Mitzvah Day has called for more initiatives to "bring the moderate voices together", saying this year's events were attacked by "far-right" extremists.
Laura Marks said the day, where many Jewish people volunteer to do charitable works, often with people of other faiths, ran counter to an agenda that was increasingly "being driven by extremes."
It comes after Press TV's Roshan M Salih claimed that a Mitzvah Day event where Jewish people taught Muslims to make chicken soup was "soft infilitation by Zionists" intended to "neuter criticism of Israel and divide the Muslim community".
Ms Marks, who also chairs the Holocaust Memorial Day Trust, told the Sara Conference at Westminster on Monday: "Mitzvah Day is about as moderate as it gets. It's all about bring people together through social action.
"Yet this year we have been hit by the far right in Muslim community and by the far right in the Jewish community because, God forbid, we brought people together...
"Most people are moderate but the agenda is being driven by the extremists."
Ms Marks added that both Jewish and Muslim women are "facing similar hatred - for different reasons".
She added: "People hate Jews for different reasons than they hate Muslims - but they hate us both because we are outsiders.
"There is an enormous amount more we can do to build alliances."
She made her remarks after Ruth Smeeth - one of the co-chairs on the conference on antisemitism against women - revealed her last few years had been "challenging" due to persistent abuse.
But the Jewish MP called for the conference to come up with solutions to the crisis, saying: "Personally I am fed up with how we have had to deal with the issue."
Former Education Secretary Nicky Morgan told the event of the "silent majority" in the UK who needed to be "unleashed" rather than the minority views of thee extremists who often dominate "the online world", she said.