In Tinker Tailor Soldier Spy (1974) the aged spymaster George Smiley is recalled from retirement to investigate whether there is a double agent, or “mole”, operating at the highest level of the intelligence service, which John le Carré calls the Circus. Melancholy and unhappily married, Smiley is drawn back reluctantly into a crepuscular world of secrets and subterfuge, where nothing is quite as it seems and even long-time friends cannot be trusted.

Making slow progress in his investigation, Smiley returns to Oxford — his “spiritual home” — to see a former colleague, Connie Sachs, who is an expert in Soviet counter-intelligence and renowned for her prodigious memory. In the BBC adaptation of Tinker Tailor, the first episode of which was broadcast in 1979, a few months after the election of Margaret Thatcher, Connie is played by Beryl Reid and Smiley with fastidious, low-toned deliberation by Alec Guinness, in one of his most celebrated roles. Their conversation takes place in near darkness, in a room lit as if by candles, like the setting for some venerable college feast.

Connie has lived through the postwar decline of Britain, which Mrs Thatcher came to power determined to arrest. She tells Smiley that her “boys”, as she calls the public school, Oxbridge-educated group with whom she used to work at the Circus, have lost their way: “Trained to Empire, trained to rule the waves. All gone. All taken away.”

In many ways, le Carré is an elegist, and the espionage novels he wrote in the 1960s and 1970s have a peculiarly sad English music — all long shadows and recessionals. His protagonists seem deeply ambivalent. They have been prepared for a world that no longer exists, and many of them are stumbling. They remain loyal to their school, college, class and, ultimately, their Queen (if seldom their wives), yet the country they serve disappoints them.

Inside the Circus, there is a feeling among the best that the institutions they are fighting to preserve might not be worth the struggle after all. And there are traitors in their ranks, those prepared to cross over to the other side — reading le Carré’s spy novels one thinks often of the shabby final years of the Cambridge spy Guy Burgess, who wore his Eton tie long after defecting to the Soviet Union.

Themes of conflicted loyalty and spoiled idealism recur again and again in le Carré’s novels and contribute to their ambiguity and fascination. Often dismissed as a mere genre writer — by Salman Rushdie, Clive James and others — le Carré is, in fact, one of Britain’s most accomplished postwar novelists, whose fiction has chronicled and continues to chronicle the great movements of contemporary history.

Apart from Tinker Tailor, his best novel is, I think, A Perfect Spy (1986), which is also his most personal book. It’s wonderfully labyrinthine, and can be read on several levels — as a detective novel, a metaphysical thriller, an anguished confession and as an experiment in forms. Le Carré uses abrupt shifts in time, as Joseph Conrad did in The Secret Agent (1907), and perspective to tell the story of Magnus Pym, an English double agent who has gone on the run after betraying secrets to the Czechs. Intelligence officers are searching for Pym, who, in turn, is searching for answers to the fundamental questions of his life: who exactly is he and what led him to betray his country? Writing in the late 1980s, Philip Roth called it the “finest English novel since the war”.

+++

On first impressions David Cornwell (le Carré was a pseudonym to preserve his diplomatic cover) seems like a typical member of the English establishment. Tall and patrician, he was educated at Sherborne and Oxford, and taught at Eton, which he called the “spiritual home” (that phrase again) of the English upper classes.

After leaving Eton, he worked for MI5 and MI6 and, because he spoke fluent German, was posted to the British embassy in Bonn. One learns from Adam Sisman in his authorised biography that Cornwell is a brilliant raconteur and mimic, and has been fabulously wealthy for decades because of the bestselling success and film adaptations of the novels. But first impressions are never entirely reliable, as any spy would know.

As Sisman tells it, David Cornwell nurtures deep resentments and class insecurities, going back to childhood. His father, Ronnie Cornwell (1906-75), was a freewheeling chancer, conman and recidivist who went to prison on several occasions. Ann, David’s first wife, called Ronnie “the only really evil person I ever met”. He had monstrous appetites — for money, women, cars, houses, always living beyond his means, never settling in one job or house for long. He hosted extravagant parties, stayed at the finest hotels (bills were mostly left unpaid), and socialised with sports stars, actors, politicians, gangsters and aristocrats. He moved from one hare-brained scheme to another, sometimes lucking out, before the inevitable fall.

One morning, when David was only five, his mother left the family home and never returned. He did not see her again until he was an adult and remained distant from her until her death. “We were frozen children, & will always remain so,” he wrote to his elder brother decades later.

At his prep school, where he boarded and encountered the usual sadistic and perverted masters, David was still wearing a nappy at the age of seven because of an inability to control his bladder. “He became especially sensitive to social nuance, noticing details to which boys from more secure backgrounds might be oblivious,” Sisman writes.

As a boarder at Sherborne David felt awkward and isolated. He was embarrassed by Ronnie, who defaulted on the fees, and by his humble relatives. He has since complained about “the indelible scars that a neo-fascist regime of corporal punishment and single-sex confinement inflicts upon its wards”. Yet, when the time came, he sent his sons away to fee-paying boarding schools, a decision he regards now as a “tragic mistake”.

After leaving Sherborne prematurely (he was 16), David went to live in Bern, Switzerland. There he read Goethe, studied German and was in tentative contact with the British security services. He completed his national service and, assisted by a contact from Sherborne (the old boy network doing its thing), won a place to study modern languages at Oxford, where he socialised with the privileged sons of inherited wealth without being one of them.

Before long he was also serving as an informer for MI5, betraying the confidences of many leftwing university friends and associates. “He had chosen loyalty to his country over loyalty to his friends,” writes Sisman. Secrecy would become a compelling preoccupation, secrecy as a way of life and as a means by which to understand character and motivation.

Sisman has written an admirable but curious biography. It’s at its best when recounting the grotesque behaviour of Ronnie Cornwell and his son David’s struggles to escape from his monstrous father’s malign influence and find purpose in life, which he did when the worldwide success of his third novel, The Spy Who Came in from the Cold (1963), liberated him to write full-time.

As the author of distinguished biographies of historians AJP Taylor and Hugh Trevor-Roper, Sisman is familiar with the mores and machinations of the high English establishment. He understands the interconnections that existed (and still exist) between the great schools, Oxbridge, Whitehall, Westminster, the Inns of Court, Fleet Street, the gentlemen’s clubs and the City. He knows the codes and can speak the language — all of which has helped in his appreciation of the textures and intricacies of what le Carré calls the “clandestine world”.

There is, however, something missing. It’s as if Sisman is, or feels, constrained: he seems unwilling to pass judgment on le Carré as he follows him on his journey through life or properly evaluate the novels. When his research contradicts something le Carré has written or told him, he simply puts it down to an instance of “false memory” and moves on. In his introduction Sisman says that his subject read the manuscript in advance of publication and that it will be revised, presumably when he is dead.

But in the book we have now, as it stands, Sisman does not really come close to capturing the inner life of the man we know as John le Carré, always the hardest task for any biographer, especially when his subject is alive. Le Carré is a man and writer of multiple contradictions. He is of and for the establishment but simultaneously estranged from it. A patriot who at university put country before friends, he has refused all official honours, including a knighthood. He has had close friendships with strident rightwingers such as the late Conservative MP and diarist Alan Clark and William Shawcross but claims to have been a long-time Labour voter (though he loathed Tony Blair). He has certainly become angrier with age, raging against the Iraq War and condemning the iniquities of “extraordinary rendition” and the rapacity of multinational pharmaceutical companies.

Le Carré has been accused of being anti-American and anti-Israeli, and has feuded publicly with Rushdie, Christopher Hitchens and Tina Brown (when she was editor of the New Yorker). He frequently changes agent, as if always restlessly seeking self-validation and a better deal (“how much am I worth”? He refuses to allow his novels to be entered for literary prizes such as the Man Booker, perhaps because he fears the humiliation of rejection by the London literati.

Sisman hints that le Carré has considered suicide but does not elaborate or explain how close he came. Similarly, we know that his marriage to Ann was destroyed by his long absences and affairs, and that his second wife has tolerated his adulteries. In particular, Ann, who had literary ambitions of her own, emerges from the book as a wounded, pleading woman. How does David feel about her upset and failures and how does Sisman feel about how she was treated? We are not told.

An outstanding absence — especially curious in a book about a major writer — is literary criticism. Sisman writes at length about the business of books: about the rights deals, agents, royalty cheques, publishers, reviews and so on. But when it comes to the novels he offers little beyond scant plot summaries. He tells us repeatedly that le Carré is a great novelist but does not attempt to explain how he achieves his effects. Who are le Carré’s precursors? What are his stylistic and technical innovations? Is he a conventional realist or a more experimental novelist? How did the spy genre evolve? What of the influence of novels such as Conrad’s The Secret Agent and Under Western Eyes (1911)? Again, Sisman has nothing to say.

In the end, one suspects, John le Carré remains a mystery even to himself. But whatever his private turmoil, his considerable public achievement has been to chronicle and interrogate the history of our times. More than this, he invented his own lexicon of espionage — the Circus, tradecraft, lamplighters, moles, scalphunters, pavement artists, the honey trap — that will endure as a permanent part of the language.

John le Carre: The Biography, by Adam Sisman, Bloomsbury, RRP£25 / Harper, RRP$28.99, 672 pages