A Board of Deputies spokesman said: "If the comments attributed to Gerry Downing about Jews are genuine, he is an obnoxious antisemite who should not be accepted in any modern political movement.
"If Labour has allowed him back in, the decision should be rescinded and an immediate inquiry launched in to how this could possibly have been allowed.
"Labour must demonstrate a clear and unequivocal zero tolerance attitude to antisemitism."
David Cameron asked Jeremy Corbyn about the readmission of Mr Downing during Wednesday's Prime Minister's Questions session in Parliament. Mr Cameron said the man must be thrown out for his comments on 9/11.
Mr Corbyn did not respond to Mr Cameron's comments.
Labour MP John Woodcock, a former Labour Friends of Israel chair, wrote to Mr Corbyn on Wednesday urging him to expel Mr Downing from the party.
Mr Woodcock wrote: "Mr Downing's stated views on the terrorist attacks of 11 September 2001 and his membership of an organisation that 'defends' Daesh's brutality are sickening and bring our party into disrepute.
"It should be noted that his disgraceful comments about terrorist atrocities were made only a couple of months ago, in January of this year.
"Allowing this man to be a member of the Labour Party insults the memory of those who died in the 11 September terror attacks and the British servicemen and women who gave their lives in the Afghanistan conflict that followed."
Labour has not responded to requests for a comment.
