The Vote Leave campaign founder then used the names of former Labour leader Tony Blair, and Mr Soros, the billionaire who fled the Nazis as a child and went to found the Open Society Foundation.
Comments about his influence in global politics are a common and growing antisemitic trope.
Audience member Richard Phillips told the JC: "I put my hand up and immediately said I knew what the name Soros meant when used in this context.
“Nick Robinson then sought to clarify my remarks to the audience suggesting this was actually about accusations of antisemitism.
“Afterwards two or three people came up to me and told me they agreed with what they said. Although interestingly one individual who I would rather not name turned around and said to me ‘You should not have said that. Daniel Hannan is a great friend of Israel etc.’
“I then said he had no right to say that to me and I argued that everybody knows what the use of the name Soros means. There is no way Daniel Hannan could not have known what he was saying.”
During the exchange, Mr Hannan insisted he was “mortally offended” by any suggestion he was using Mr Soros' name in a context which could deemed to be antisemitic and insisted he was actually a philosemite.
But on Twitter, another member of the audience who had also attended the debate, which was organised by Intelligence Squared, wrote:” Disappointed but somehow not surprised at the shoe-horned 'George Soros' dog-whistle I heard tonight from Hannan.
"Do you really not know what this means? It's easy enough to find out.”
Mr Phillips, who is himself Jewish, added:”The whole story of George Soros is interesting – he gets blamed for everything. He’s the guy who finances Open Democracy movements all over the world and I’m a great fan of.”
Mr Hannan had earlier this year appeared at a joint Board of Deputies and Jewish Leadership Council event as a panellist discussing Brexit. He accused the EU of having a “disproportionate focus on Israel.”
Mr Hannan’s comments at the Intelligence Squared debate on Monday echo an article he wrote for the Washington Examiner on the same day in which he said: "If anyone wondered whether there was still such a thing as the British Establishment, no doubt can now remain.
"In the two-and-a-half years since the referendum, civil servants, politicians, financiers and politically-connected business cartels have worked assiduously to overturn to result.
"It wasn’t a conspiracy — though we might quote H.G. Wells and call it “an open conspiracy,” in the sense that its agents never hid what they were doing. Some, including George Soros and Tony Blair, sought to overturn the result outright with a new referendum. “
The JC has contacted Mr Hannan and Mr Robinson for comment.
Pro-Brexit group Leave.EU is also being criticised for using a picture of George Soros in an online campaign which suggested he was a member of a “powerful elite” that were out to “openly mock the rest of us.”
The campaign tweet used an image of Mr Soros with a quote from Maria Caulfield, the Tory MP for Lewes, who said: "Not sure there is even democracy in this country anymore. The powerful elite seem to know best and openly mock the rest of us.”
Ms Caulfield responded, demanding they remove it, saying she "completely denounces" it and it was branded "shocking and disgraceful".
