Experts have called on the Football Association, Premier League, clubs and footballers themselves to do more to tackle antisemitism in the game.
Panellists including football writers Henry Winter and Anthony Clavane as well as Tony Bloom, chairman of Brighton and Hove Albion, also agreed that using the word "Yid" was unacceptable, even by Tottenham Hotspur fans using it as a club nickname.
They made the comments at a discussion which attracted around 200 people to the JW3 community centre in north-west London, which followed news from anti-racism group Kick It Out about a rise in reports of discrimination for the fourth season in a row.
Audience members expressed concern at the lack of action taken by the authorities with the panel citing incidents such as former Premier League player Nicolas Anelka's antisemitic quenelle gesture in 2013.
Mr Winter, chief football writer at The Times, said: "We have warped priorities in our sport.
"There is a slight element of the FA shining a light on an issue then moving on. The key thing they have to do is have constant vigilance, and unfortunately you're looking at an organisation that's just cut 100 staff - and Kick It Out only has 13 staff.
"Maybe we should be calling the Premier League to account, because they are sitting there with billions in the bank."
Mr Winter said he had called the FA before attending last Thursday's event to ask for statistics about antisemitism in the sport.
"They reckoned there was a decrease," he said. "I go to so many games a year, and I think that's rubbish."
Roisin Wood, director of Kick It Out, agreed authorities needed to take action, adding her concerns about social media, which she said represented the biggest section of the group's work.
"Of our reports, 32 per cent are about social media discrimination. It has to be taken seriously, because it's not going away. It's just getting worse.
"My plea is to the police, to train their officers around it," she said, recalling how one officer had asked her what Twitter was.
Jonathan Metliss, chairman of the Action Against Discrimination group which hosted the event, criticised prominent figures in the game.
He said: "I invited Spurs chairman Daniel Levy to come tonight and he didn't reply. I asked West Ham co-chairman David Sullivan to attend, but he gave a large amount of money [as sponsorship] instead.
"We also invited the FA and they accepted the invite and paid for a ticket, but I don't think they are here."
Mr Metliss, a prominent lawyer and Chelsea fan, told the audience he was against Spurs fans using the "Y-word" at matches.
Anthony Clavane, a Leeds United fan who wrote Does Your Rabbi Know You're Here? - about the historical connection between the club and the Jewish community - agreed. He said: "The people to be blamed are the ones at Chelsea and Leeds who sing songs about Hitler and Auschwitz and use the Y-word to abuse Spurs fans, but I also feel very uncomfortable with Spurs fans reclaiming that word."
He added: "I was being interviewed by a football club chairman and, referring to someone else, he said: 'He's a yiddo, like you'. I said I was not a Spurs fan, and he said, 'no, I didn't mean that'."
On the pitch, he suggested extreme action to draw attention to discrimination. "Players should walk off when the crowd is being abusive, and the whole team should aswell, to show solidarity."
Mr Bloom said he was "not necessarily an advocate of walking off unilaterally, but would "totally understand if a Jewish player did".
Joe Jacobson, a Jewish professional footballer who plays for League Two side Wycombe Wanderers, said education was crucial. The 29-year-old said: "Many footballers don't even know that saying the Y-word is a bad thing. They just think it's something you say at a football game."
He said he had never personally suffered from antisemitism. When team-mates discussed his Judaism it was an opportunity to open players' eyes.
"When I'm in my close friendship group in the locker room, I get a lot of things thrown at me about being Jewish, and I throw some stuff back, and by speaking to these people they end up being more educated.
"A lot of the boys I play with say I'm the first Jew they've ever come across, and that's probably not true - I'm just the first one they know about."