In 2012, Ms De Toledo was warned by the umpires for steering too close to the Cambridge crew. This was after the race had to be restarted due to protester Trenton Oldfield, who had jumped into the water and almost hit the boats.
After the restart, and despite attempts to move away, the boats clashed and Oxford number six Hanno Wienhausen lost his oar.
Worse followed as Oxford rower Alex Woods - now Ms De Toledo's partner - collapsed from exhaustion at the finish line and was rushed to hospital.
Then, two years ago, Ms De Toledo went through a divorce, and she was left homeless last year when a fire destroyed her home in Hambleden, north of Henley.
Mr Kaye, a member of Kenton Synagogue, explained how his granddaughter recovered: "Zoe has had some difficult times but she has pulled through as she has such a strong character. That's what makes her so suited to this role in the team.
"It wasn't long before she was back out on the water again at the crack of dawn. The thing she prides herself on is that she remains calm and does not get overawed by the occasion. And this final won't faze her one bit."
Following Alice Schlesinger's exit in the judo, Ms De Toledo is the only remaining Jewish Team GB competitor.
The 28-year-old, who is 5ft tall and weighs under eight stone, has been involved in rowing since she was at St Paul's Girls' School in Hammersmith, West London.
She started swimming aged four but took up rowing when the school introduced a boat club, eventually going on to compete for the prestigious Leander Rowing Club.
Paul Budd, the general manager at Leander, said: "You need to make big sacrifices and Zoe, like her team, would have done that.
"Zoe will be used to the early starts having to be up at 5.30am for three sessions a day, six days a week. It's not easy on a winter's morning, when you have to get into a cold boat. But that's where the medals are won and lost. And you can forget about late nights and pubs.
"You also have to watch what you eat. Zoe has to stay light. You need dedication and she has always possessed that in abundance. Out of the boat, she is a vivacious and friendly person but, in it, she has a steely-eyed determination."
Mr Budd added: "It can be tough for female coxes. If you are a male cox who is aggressive and cocky, those are seen to be good attributes. If you are a female cox with those same attributes, people might not take to you, which is wrong.
"But Zoe has encountered no problems, which says a lot about her as a person and a professional."
Ms De Toledo will be cheered on by her by father, Tedi, mother, Laine, and step-dad, Magnus, in the 2,000m race for gold. It will be her final major race before the postgraduate psychology student returns to Oxford University to study medicine.
Ms De Toledo's cousin, Justin Kett, said: "We are a close family and will be glued to our TV screens watching. There is WhatsApp group set up where we are all in regular contact.
"Everyone is so excited. Zoe's 13-year-old cousin Joshua is on RSY camp in Wales but I'm sure he will be doing his best to tune in."
