Lloyd Dorfman never had a long-term plan. By his own admission, he "just did" and things came through. Now worth an estimated £550 million, it is an attitude that obviously paid off for the self-made businessman and philanthropist.

But Dorfman, 63, founder of Travelex, the world's largest currency exchange business which marks its 40th anniversary this year, was not always one of the big players. As a young entrepreneur in a market dominated by large banks, he had to work seven days a week and persistently fight to re-open doors that were dismissively shut in his face. Despite being recognised for Travelex, of which he still owns a five per cent stake, Dorfman sees himself an entrepreneur, not a foreign exchange guru.

"People think I am a foreign exchange expert, but I am absolutely not," he says. "I am just a builder of businesses. It just so happened that I decided to build that business. One man asked me what I thought was going to happen to the pound in the next few weeks… like I knew," he laughs.

So was Dorfman - now chair of flexible office space company The Office Group and retail collection group Doddle - always ambitious, or just lucky? I have come to Dorfman's central London office to find out. He walks into one of the ground-floor meeting rooms, confident, 6ft 3in,bespectacled and warm, if a little suspicious: "You never know how an interview is going to come out or what the tone will be until the thing appears."

So what has been the secret of his success. "I just did," he says. "It was 'ready, fire, aim' rather than 'ready, aim, fire'.

You never know where the next geek's big idea will come from

"Can I produce the 40-year business plan that will show you, from bottom-left to top-right, how my one little bureau de change shop was going to become a global brand, a leader in its field? In the real world of building businesses, that is not how it happens.

"In truth, it has been a commercial venture, the like of which I never imagined. Today, I sit with governors of central banks and prime ministers and presidents and finance ministers and they all want Travelex in their airports. It's been an amazing journey." Even in conversation, it's clear that Dorfman, who was made a CBE in 2008 for services to business and charity, has no structured method of thinking - indicative perhaps of his instinct-led way of doing business, and doing it so well.

"My mind works on a bit of a radar basis," he concedes. "I do not think I would be very good at playing bridge. My wife persuaded me to take one lesson once. I spent two hours in that bridge lesson and thought: 'I can't focus on one thing like this for two hours'."

Perhaps that is why, aged 24, he decided to leave his stable job in the City to set up a small foreign exchange bureau with a £25,000 loan from a family friend. Anticipating the arrival of tourists ahead of the 1977 Queen's Silver Jubilee, Dorfman thought it would be a good idea to open his shop on Southampton Row near the British Museum - despite foreign exchange then being an "industry dominated by some of the oldest and largest banks in the world - and there was already talk of a common European currency."

So why would you start a money-changing business then?

"Because sometimes you can think too hard about these things. Just do it and when you do it, stuff happens. I decided to give it a go.

"It was tough to break in. It was a time where the banks felt they had to provide every service and be the supermarket of financial services however small or big they were themselves.

"What I was looking to do was provide a money-changing service seven days a week. I was going to specialise in one particular - and in truth core - activity of the big banks.

"For the first three months, it was just me. I used to have to shut the shop to go to the toilet in the hotel next door."

The business swiftly grew and Dorfman received his big break after securing a contract at Heathrow Airport in 1985, after months being shunned by the big boys.

"They were totally dismissive," he says, noting that he still has the "wonderful sort of patronising, slightly snotty letters'' he received at the time.

Dorfman clearly has a flair and passion for business, a trait that he did not extend to his studies. A former St Paul's student, he rejected university unlike most of his peers. "Originally, I thought I would take a bit of time off after school and then become a barrister - but then I decided it wasn't for me. Writing essays and reading stuff - no, my one love was always business."

For one year, he committed to his law studies at Lincoln's Inn, where he met his future wife's sister - a barrister who proved to be the couple's matchmaker. Newly engaged, his future father-in-law suggested he join him in the City. Ironically, Dorfman now sits as an honorary bencher of Lincoln's Inn.

That time in the City was an invaluable business apprenticeship: "It was the end of 1973. Within a matter of weeks there was a quadrupling of the oil price overnight, there was a Middle East war; there were strikes, three-day working weeks, a stock-market collapse. It was a real learning experience.

So what is the trend that links his successes? "The things I do in business tend to be challenger businesses - against the established mode."

But now, he's one of the establishment himself. And, having spoken to some of his close friends - "legends in the property business" including Gerald Ronson, Sir Stuart Lipton and John Burns - he's become a property mogul.

There was a time when he admits he had to cold-call for opportunities but now, he says: "When you are at my stage of the career, I can pick up the phone and have a conversation with people that is more difficult if you're 25 and starting out.

"Over the passage of time, you come to know people and it does help. But in the early days you've just got to be persistent and think a bit differently: What is your edge? What is your USP? And just hang on in there."

Chairman and major shareholder of the Office Group, which boasts one million sq ft across its portfolio of 32 buildings in London, he says it is not your typical property business. "It's challenging the conventional way - it's pioneered and led the way for office space and co-working today."

And he says, it reflects the corporate bodies needing the young entrepreneurs - rather than the opposite, as it was in his day. "Our clients aren't just start-up companies, they are also large corporates and multinationals, including Facebook and Santander.

"In the past, a lot of the small companies wanted to be nearer the big ones - but now it's the other way round. Corporates want to be near the start-ups, it's about the high energy and creative things they do.

''Just look at the genesis of Facebook and Twitter - all the big digital leading companies today were started by these guys in their garages doing their geeky thing.

"I think a lot of the big corporates realise they should be embracing this different way of working. They've learnt that they need to stay close to these creative pools of energy because you never know when the next geeky guy is going to come out of his garage with the next geeky idea which will transform the whole industry."

Away from work, Dorfman doesn't favour one political party over another, though admits he'd find it "difficult" to support Labour in its current form, noting that the rise of antisemitism within its ranks "is a big concern. It would be very difficult. I think a lot of Jewish, Labour friends, peers of mine, are finding it very difficult."

Dorfman is also deputy chairman of the CST and a JW3 trustee.

Enthusiastic about the arts, he was announced as one of the main backers of the new 900-plus seat theatre being set up by Nicholas Hytner and Nick Starr, former executive director of the National with whom he has worked in the past, near Tower Bridge in London.

And arts philanthropy is key, for the father of three and his family (his wife is a governor of the Royal Ballet School) .

"We do it brilliantly in this country -we are world class at it. It is at the heart of our nation's culture and soul, it brings entertainment, makes people think."

Charity is another area where Dorfman - chairman of the Prince's Trust and Prince's Trust International - thinks the Jewish community does more than its fair share.

"With the Jewish community, " he says, "it's part of our DNA. We punch well above our weight." He also notes that, in addition to the National's Dorfman Theatre, the Clore Learning Centre and Max Rayne Centre were all named for members of the West London shul.

"I do think it's interesting that not only are those three Jewish names, they also happen to be members of the same synagogue - West London."

With that, Dorfman bids me farewell and runs off to his next appointment, ever-eager for the next deal.

To me, it is clear that, while luck and ambition may have had a part to play, it's clearly passion above all that has made Dorfman such a success.