The pleasure of the annual Booker prize jamboree is its seemingly endless capacity for surprise. If the leaks and gossip from inside the judges’ camp were to be believed, then this was to have been the year that Salman Rushdie became the first author to win the prize twice; if not him then his compatriot Vikram Seth, or Roddy Doyle, or some other “big name”, in the tiresome argot, on whom a story of instant journalistic convenience could be hung. In the event, another former winner, JM Coetzee, emerged as the most likely recipient of the 31st and final Booker of the century. This would be appropriate recognition for a writer of the highest creative intelligence, for whom the absurd, labyrinthine politics of postwar South Africa have inspired a body of work of almost unrivalled complexity and fascination.

That Coetzee will not even be at the Guildhall on Monday 25 October to accept his award - if indeed he wins, as he surely should - is entirely in keeping with the little that is known about him. For he is a writer who, like Beckett before him, spurns literary celebrity, living quietly in Cape Town, where he is professor of general literature at the main university, seldom if ever giving interviews, and adhering to an austere working routine. In person, the slight, silver-bearded Coetzee is reticent, wary. Friends speak of his “devotion to privacy”, his minimal conversation, his untrusting gaze. He is divorced and has one adult child (a son was killed in an accident at the age of 23); he is keen on rugby and cricket and on vegetarian cooking.

“Coetzee,” says the writer Rian Malan, “is a man of almost monkish self-discipline and dedication. He does not drink, smoke or eat meat. He cycles vast distances to keep fit and spends at least an hour at his writing-desk each morning, seven days a week. A colleague who has worked with him for more than a decade claims to have seen him laugh just once. An acquaintance has attended several dinner parties where Coetzee has uttered not a single word.”

So clearly, the work is the thing for Coetzee, and nothing but the work. And what work. The Booker-shortlisted Disgrace (Secker & Warburg, £14.99) is his eighth and arguably best novel. A parable of the new South Africa, it is written in prose stripped of all superfluous ornamentation and offers a portrait of a society terminally wounded by the sins of the past and of the present. David Lurie teaches communications at the Cape Town Technical University. He is twice divorced, jaded and fatigued by post-apartheid South Africa, with its double-think, violence and affirmative action projects. In his spare time, Lurie uses prostitutes and works on the libretto for an opera about Byron.

When the prostitute with whom he enjoys weekly sex quits the game, Lurie stumbles with the certainty of a sleepwalker into an affair with one of his pretty dark-haired students. The affair goes wrong; Lurie is reported to the authorities at the university, called before a disciplinary committee and dismissed when he refuses to apologise publicly. From there he travels out to the Eastern Cape to visit his daughter, Lucy, on her isolated homestead, where she keeps a kennel of dogs and struggles to survive in hostile terrain. One afternoon, three black men raid the homestead. They attack Lurie, setting him alight, kill the dogs and gang-rape Lucy, who then angers her father by refusing to report the violation to the police. In her own mind, what has occurred is bound up with the wider historical guilt of whites in South Africa; this is her burden as a white woman.

Lurie is a misanthrope and pessimist. Even after the attack on his daughter, even after the father of the student whom he has abused confronts him, Lurie continues to use prostitutes. He is a man, you realise, who has learnt to live without the possibility of delight, a man alone. He himself accepts that his pleasure in living has been snuffed out: “Like a leaf on a stream, like a puffball on a breeze, he has begun to float towards his end. He sees it quite clearly, and it fills him with (the word will not go away) despair.”

Disgrace is an extraordinarily bleak novel, but one of terrific power. Coetzee’s spare prose is superbly readable and Lurie’s fall into self-knowledge has a raw truth. Anyone who has recently visited South Africa, as I have, will also respond to his uncompromising portrait of that country. For all the talk of renewal and opportunity and the abolition of the race laws, for all the supposed cathartic work of the Truth and Reconciliation Commission and the rhetoric of the “rainbow nation”, South Africa is not a nation at ease with itself. How can it be at ease when whites, moaning bitterly about the deleterious effect of affirmative action, lock themselves away in their gated communities, with “armed response” signs slapped to the front of their houses like estate agents’ banners; when the coloureds I met complained of once not being white enough and now not black enough to progress; when most blacks themselves are poor and illiterate and live in shanty towns, smouldering with resentment; when the ANC seems set to become nothing but an elected dictatorship? Coetzee captures this pervading unease and much more. He is, in every sense, the ideal chronicler of the new South Africa: clear-eyed, hard-headed and a scourge of sentimentality.

John Michael Coetzee was born in 1940, the son of mixed Afrikaner and German parentage, and one suspects he has long felt like an exile in his own land. He grew up in Cape Town, speaking two languages, English and Afrikaans, but came from a resolutely Anglophile family that felt estranged from the supremacy of Afrikaner culture.

When Coetzee was eight, his father - after losing his government job because of his “progressive” views - moved the family to the oppressive provincial town of Worcester, and he has spoken of a “time of gritting teeth and enduring”. In his memoir, Boyhood: scenes from provincial life, Coetzee, writing about himself with characteristic reticence in the third person, as if his younger self were one of his own creations, describes his sense of alienation from fellow Afrikaners and from the laws that separate the people around him into racial sub-groups, when so many of them, palpably, are anything but pure. In fact, they are what Breyten Breytenbach, the Afrikaner poet and political rebel, famously called “bastards” - racial hybrids, formed by generations of migration and wandering.

Coetzee was a talented student, particularly in mathematics and linguistics. He lived and worked for a period in England, in the 1960s, before he took his PhD in literature at the University of Texas at Austin. He taught at Buffalo University before returning home, in 1972, to join the literature department of Cape Town University, where he has remained ever since. His first novel, Dusklands, an oblique commentary on the Vietnam war and on the arrival of Dutch settlers in Africa, was published in 1974.

Leading South African writers are becoming increasingly obsessed by the violence, the moral dislocation and wilful cruelty around them. “This has always been a violent country,” writes Breyten Breytenbach in Dog Heart (Faber), his account of his recent return from exile to travel in the Western Cape. Dog Heart burns with the anger of Afrikaans-speaking whites and “coloureds” who have been attacked, robbed and raped in the new lawless South Africa. Nadine Gordimer, too, in her most recent novel, The House Gun (Bloomsbury), has written astutely about post-apartheid violence in her parable of a white man who murders his lover and then is defended in court by a black lawyer.

JM Coetzee is a more subtle writer than the politically explicit Gordimer, preferring to work through allegory and parable, perfecting a kind of prison literature: his lonely characters (like the eponymous hero in Life and Times of Michael K, which won the 1983 Booker) operate in societies without any recognisable moral centre, often afflicted by a nameless menace, abused by functionaries of an impenetrable state, guilty of no sin except that of being alive. So life is a sort of prison sentence; birth is a crime.

In Michael K, Coetzee describes the bewildering journey that a young, simple-minded coloured undertakes with his ailing mother across the Cape. Michael K, like Kafka’s K on whom he is modelled, is a holy innocent whose inarticulacy mimics that of the wider inarticulacy of the once-oppressed majority in South Africa.

Coetzee has written about voicelessness before, in his novel Foe (1986), a smart rewriting of Robinson Crusoe, in which a woman called Susan Barton returns to London with a mute black man (his tongue has been cut out). She meets an irascibly famous writer, Daniel Defoe, to whom she tells her fabulous story of having been marooned on an island in the middle of the Atlantic with two men, Crusoe and Friday.

As a novelist who runs a dual career as an academic, Coetzee is intensely interested in narrative theory, and his fiction is deftly experimental, if at times mechanistic and overdetermined. In Foe, for instance, he intervenes in his role as the controlling narrative intelligence to speak on behalf of the mute Friday, to give him a voice and his own cherished story.

These metafictional devices are not always successful - the second part of Michael K consists partly of extracts from the diary of a camp doctor who meets K but who also offers an unnecessarily over-elaborate summary of his life and the themes of the novel, just in case the reader hasn’t already got it. In another work, The Master of Petersburg (1994), Coetzee even has Dostoevsky as a hero, reworking an episode from the writer’s life, as he did with Defoe, in which the murder of his stepson Pavel (which never actually happened) inspires the genesis of his great novel of political upheaval, The Possessed.

Coetzee has said of Robinson Crusoe that the idea of a man being marooned alone on an island is perhaps the “only story”; and certainly there is in his work a pervading sense of rootlessness, of inner exile, of shipwreck being embraced as an ontological condition.

So what is Coetzee’s vision of the future of South Africa? He has long resisted making public pronouncements, to reducing his work to a series of political or revolutionary slogans, either for or against the system. Unlike, say Breytenbach, who was imprisoned for seven years for working on clandestine missions for the ANC (an experience he wrote about in his great prison memoir, Confessions of an Albino Terrorist), Coetzee was never an overt activist. He has spoken, too, of how people living in the benign liberal democracies of the west will never acquire an understanding of the “intensity of life in certain critical situations”, its tragedy and horror. But in Disgrace, his most nakedly political book, there is a clue to his world view, and it is extraordinarily despairing. Lucy, contemplating the wreckage of her life after the rape, says that she is prepared to start again, “with no cards, no weapons, no property rights, no dignity”. “Like a dog,” Lurie says. “Yes, like a dog.”

The phrase returns us to Kafka, as so much of Coetzee does, and in particular to the dying Joseph K, in The Trial, who looks up as a knife is being thrust into his chest by an anonymous man.” ‘Like a dog!’ he said: it was as if he meant the shame of it to outlive him.”

South Africa is a country of deep shame, but no shame will ever attach itself to JM Coetzee. There is no more appropriate writer to become the first double winner of the Booker, even if he has decided to stay away from the Guildhall on what one hopes is his night of glory.