While I nurtured vague ambitions to be an author as a young man, I made sure I combined impactful life experiences with learning the craft of journalism. After volunteering to help London’s homeless, I backpacked around Europe, taught myself Spanish in Chile, and worked in Nicaragua towards the end of the Sandinista regime.

This was my apprenticeship. Or so I assumed. Yet while I was accumulating what I thought were valuable insights, and trying to earn a living wage from journalism, the fashion in literature changed. Since the publication of The Remains of the Day and Atonement in the 1990s, the most celebrated novelists have published, almost exclusively, works based on detailed historical research. The trend has continued, with recently garlanded books such as The Essex Serpent and Lincoln in the Bardo, while On Chesil Beach gets the motion picture treatment.

It is quite a contrast with previous generations of novelists, who drew heavily on their first-hand experiences. Cervantes fought in a war and was a galley slave. Charlotte Bronte used her time in Belgium to craft Villette; George Orwell visited coal mines, lived as a tramp and fought the fascists in Spain. Then there were the war poets ….

Probably the main reason that today’s more sheltered writers do not use their own lives as research is that they have been rather dull; the books would consist of tales about going to literary festivals, using the internet for research, remembering to patronize independent coffee shops, and feeling annoyed about not being shortlisted.

Now, at its best, historical fiction can combine a thrilling plot with insights into life in former times. At its worst, the result is rather passionless prose in a work that is non-fiction in disguise, with gratuitous historical notes inserted clumsily into direct speech, resulting in ‘dialogue’ such as:

‘How’s business?’

‘Looking up! This Enclosure Act will help boost farm productivity.’

‘We’ll need it. I hear there’s revolutionary trouble brewing in France.’

‘And America too, according to my cousin. You mark my words: they’ll be fighting for independence next!’

OK, the above is a caricature, but I have come across sections that are almost as bad.

I do believe something has been lost in the almost pathological distaste for drawing upon one’s emotions and life experiences. By definition, history has already happened, whereas a writer’s own life has not. It is new, and unique.

I wonder whether ghostwriting – the practice of partnering someone who has had a bit of a life with a professional author – might produce more vivid literature. The right pairing can produce more than the sum of the parts. My proudest review of a book for which I had been hired as the professional scribe occurred when the reviewer suspected that my client, a successful entrepreneur, had employed a ghost. The review referred to the obvious input of a top financial journalist. Yet I do not have a financial background; the insights into the economy came from my client, who had an economics degree. The analysis and composition attributed mostly to one person had actually resulted from the close collaboration of two of us.

Recently The Guardian, critiquing the comparatively minor trend of auto-fiction – memoirs-based novels such as My Struggle by Karl Ove Knausgaard – asked: Why have novelists stopped making things up? It was the right headline for the wrong article. No, don’t do that, Guardian! I muttered to myself (a writer’s quirk), splurting out my coffee. Don’t nip this encouraging development in the bud, please! Better a bit of passion from real life than the dull, decaffeinated prose of the meticulous historical tome.

Sadly, I see little end in sight to the trend for historical fiction. If you want insights into the times we are living through, you’ll have to look to biographies, often ghosted. And my novels, too, of course.

• Philip Whiteley is an experienced author of non-fiction and fiction, and a ghostwriter. He writes novels under the byline PJ Whiteley. More at pjwhiteley.com