Simon Garfield has written a book about his passion for philately.

Simon Garfield has a confession to make. He's been having an affair. But not with another woman (although that is true as well). His clandestine passion is less about midnight tristes and steamy sex, and more about visits to the Post Office, because the London-based author has an obsession with stamps.

Garfield acknowledges this could be perceived as weird - he admits that it was more embarrassing to write about his fixation with philately in his new book The Error World: An Affair with Stamps, than about his extra-marital affair.

"If you go to the bookshop, you'll find there are lots of memoirs and autobiographies where people talk about the break up of their marriage, but there are very few about stamps," he says.

It is true. Stamp collecting does not have much sex appeal and Garfield is not trying to change the image of the nerdy guy holed up in his bedroom with a magnifying glass and a pair of tweezers agonising over the perforated square piece of paper that will complete his definitive series. Instead he celebrates it.

"It was an attempt to write a memoir with stamps as a narrative throughout." says Garfield, who is known as a writer on pop music. "I was able to share my enthusiasm because it doesn't have the coolest image in the world. I'm not sure my book will do much to change that. I always like talking to people who are passionate about anything. The English do that passion for hobbies and I love all that. I'm 48, I'm still reasonably in touch with modern music - my kids keep me young and I love going to gigs and stuff and I haven't entered an old nerdy world. "

Garfield has previously written books on niche subjects like wrestling, Radio 1 DJs, and how the colour mauve was created, but he is perhaps best known for his three-volume anthology of the diaries of ordinary people during the Second World War.

Error World is Garfield's first autobiographical work. His hobby is an excuse to tell anecdotes about the quirky world of furtive collectors, auction houses and suppliers. "Postage stamps offer one way in which we can order a world of chaos and they have the power to bring a dependable meaning to a life," writes Garfield, who lost his father to a heart attack when he was 13 and when he was 20, his mother died of cancer. Before then he says he enjoyed a "reasonably ordinary but typical life of a Jewish boy". He lived in Hampstead Garden suburb, in North London, was educated at University College School and went to the local synagogue in Norrice Lea.

Like many others, Garfield's stamp fixation started in boyhood and faded away when he was in his twenties. He revived the passion in his forties, and admits spending thousands of pounds building up his collection of "errors" - the relatively rare stamps which have been mistakenly altered during the printing process.

Stamp collecting may have had a link to his break-up with Kindertransport playwright Diane Samuels, with whom he has two sons, Ben and Jake. But the author is sensitive enough not to divulge too much information.

"It's not a kiss-and-tell kind of thing at all," he insists. "The only person exposed is me. I wrote it during and after the split so we were at marriage guidance. The counsellor asked a lot of questions that I maybe found painful but having done that, writing about it was a very cathartic thing."

But more than stamps or his love life, Garfield says the hardest part was opening up about his brother Jonathan whom he lost the year before his mother to a rare type of pneumonia. "My brother dying when he was 23 was such an unnatural thing. A lot of people have parents who died when they were in their teens. But it was almost impossible for other people to take it all in when my brother died. Their reaction was so shocked and horrified I was protecting myself. I did find it quite hard to write."

Given this background, it is understanable that Garfield would have turned to stamp collecting for constancy and order. But it is also understandable that the women in his life do not take an interest. Error World mentions only one female stamp collector. He admits the "other" woman - Annie, a childhood friend - is not keen on his collection, and his marriage guidance counsellor was not impressed with his album.

His relationship with Annie was "passionate and destructive" and eventually he left her for someone he met 14 months ago. While he says he is very much in love with Justine Kanter - with whom he lives in Hampstead near his ex-wife and two sons - it is not because of their shared love of stamps.

"It's wrong to say women don't collect because they do," he argues, admitting that it is a less common pursuit among females. "Maybe they have got their priorities right, especially if you have a marriage and kids. It's also an excuse to get into a specialist world. Maybe woman don't need that or can't do that. Maybe it's a luxury for men to go off on a Saturday to a collectors' fair.."

Incidentally Garfield also has a collection of Chelsea FC badges, London Underground maps and Corgi model cars. "There are far more obscure collectors than mine. I'm convinced - especially after seeing eBay, that there's nothing that a person can collect without someone else collecting it too."

The Error World: An Affair with Stamps is published by Faber, £14.99