Mr Willsman was standing for re-election to the party's ruling body on a slate of nine candidates backed by Momentum, the left-wing pro-Jeremy Corbyn group.
But the group has ditched him, calling his comments "deeply insensitive".
Listen to Pete Willsman's rant below:
Mr Willsman apologised privately to party General Secretary Jennie Formby and then again publicly after the recording was published by the JC. He also said he would undergo equalities training.
But on Wednesday evening, two days after the recording surfaced, Momentum tweeted it had decided to withdraw its support for Mr Willsman and urged people to vote for its remaining eight candidates.
The elected officers of our National Coordinating Group have decided to withdraw Momentum's support for Pete Willsman in Labour's ongoing NEC election.
"While it is welcome that (Mr Willsman) has made a full apology and will attend equalities training, his comments were deeply insensitive and inappropriate for a Momentum backed NEC candidate," they said.
"We must also acknowledge the anger and upset felt within the British Jewish community and reaffirm our commitment to rooting out antisemitism both in the party and across society."
During his rant at a July 17 meeting of Labour's NEC, Mr Willsman said the 68 rabbis who wrote an open letter warning of antisemitism in the party should produce evidence.
In the recording, he also uses the phrase "falsify social media", implying antisemitic posts written by Labour members could be fake.
