Former Mayor of London Ken Livingstone has reflected on his fraught relationship with British Jewry in his newly-published memoirs.

In You Can't Say That, Mr Livingstone details long-running disputes with the Board of Deputies, explains his views on Israel, and recalls the infamous evening when he told a Jewish journalist he was "just like a concentration camp guard".

The 66-year-old, who hopes to be re-elected as mayor next year, revisits disagreements dating back to the 1980s and his then role as Greater London Council leader.

In early 2005, Mr Livingstone felt "everything was going right" at City Hall except for minor complaints about the Olympics and the "grief" he was getting from the Board.

He recalls how when a Board delegation visited him he "didn't know what to expect. Coming straight to the point, they asked me to tone down my comments about Israel and gasped when I said I was already doing so."

Mr Livingstone said he explained to the Board delegates his opposition to Israeli policies and told them "it would be easier to achieve peace if Israel comes to terms with the crimes committed at its birth".

When the meeting "resolved nothing" he chose to turn his efforts to working with the London Jewish Forum in order to "concentrate on issues where we could work together to improve life for London's Jewish community rather than endlessly debate the Middle East".

It was the incident in February 2005 with Evening Standard reporter Oliver Finegold that significantly damaged Mr Livingstone's relationship with the Jewish community.

Mr Livingstone writes that "as the phrase 'behaving like a concentration camp guard'…is a common jibe in Britain, no journalist had ever complained before and I forgot about the matter". But when the transcript of the clash was leaked to the Guardian, the Board again attacked him.

In a Today radio interview, then Board president Henry Grunwald QC"linked my comments to the rising tide of antisemitic incidents in Britain…and wailed 'can nothing be done about this man?'."

Despite widespread criticism from political opponents and the media, and Tony Blair's request that he should apologise to Mr Finegold, Mr Livingstone remembered at the time: "I have lost count of the number of times I have been approached by Londoners over the last two weeks and have been urged very forcefully not to apologise."

But he added: "My words were never intended to cause such offence [to Holocaust survivors] and my view remains that the Holocaust against the Jews was the worst racial crime of the 20th century."

Israel's war with Lebanon in the 1980s caused other friction. Following the deaths of thousands of Palestinians in the Sabra and Chatila refugee camps, the Labour Herald weekly newspaper – which Mr Livingstone helped edit – depicted Menachem Begin in a Nazi outfit standing on a pile of Arab bodies.

Mr Livingstone remembers: "We were denounced by the Board of Deputies but this did not stop us campaigning to get the Labour Party conference to recognise the PLO as the sole legitimate leadership of the Palestinian people."

He cites the 1993 declaration between PLO leader Yasir Arafat and Yitzchak Rabin as proof that his efforts a decade earlier had been well founded.

There was, however, one area in which he felt successful co-operation had been achieved with the Jewish community: his work with the Board to establish a Holocaust remembrance ceremony at City Hall.

Mr Livingstone wrote: "I found it particularly moving to see Holocaust survivors standing side by side with schoolchildren who were often learning about these events for the first time."

Mr Grunwald said: "Mr Livingstone was fixated about the Board which some may say was not a bad thing, because it shows we had an effect.

"I don't remember 'wailing' but I may have done because everyone was upset. I won't be buying the book."