After his arrest, David, who also paid his children's private school fees with the cash, told police: "I am public enemy number one in the Jewish community and understandably so. I have very very few friends left and I understand why."
David stole a total of £14,545,594 from a total of 55 clients over ten years.
He admitted fraud charges and was jailed for six years at Southwark Crown Court in 2018.
David is now facing a confiscation hearing in which the prosecution will probe his assets in a bid to claw back the missing millions.
But David refused to get into the van at Ford open prison in West Sussex - and he may now be moved to the harsher regime of HMP Wandsworth.
Sophie O'Sullivan, defending, said: "His decision to not get in the van was a very specific one.
"It is known that when people come here over two days they go back to Wandsworth.
"He is anxious of not being able to return to Ford.
"Ford allows him to prepare his food because of the open facilities, and by preparing his food he can adhere to his faith.
"He did not wish to not attend today."
But Judge Deborah Taylor said: "This court has dealt with many Orthodox Jewish defendants and it has not been an issue before."
Judge Taylor postponed the hearing until Thursday.
David, of Hartfield Avenue, Borehamwood, Hertfordshire, admitted one count of fraud by abuse of position and one count of obtaining a money transfer by deception.
He was jailed for six years and disqualified from being a director of a company for 10 years in July 2018.
