Most of us have a year of great personal or political moment, which, for whatever reason, resonates peculiarly, so that something like a chance hearing of a half-forgotten pop song or a casual newspaper reference to distant events can return us to the sights, sounds and textures of a world of yesterday. To me, there has always been something extraordinarily memorable about 1977, the year of the emerging punk and disco scenes, of the Silver Jubilee celebrations and the “Yorkshire Ripper” murders - something bound up with the country that Britain once was, and irreversibly became.

I’m not alone, it seems, in finding special resonance in 1977; in the past 12 months or so, there has been a curious (and coincidental) confluence of books and films that use the events of this year as the starting point for a series of imaginative retrospectives, exploring themes such as alienation, social dysfunction and the end of a certain kind of England. The best of these include John King’s novel Human Punk, which is about a young DJ from the London suburbs rendered unstable by violence and boredom; David Peace’s Nineteen Seventy Seven, a fictional recreation of the north of England during the time of the Ripper murders; and Whatever Happened to Harold Smith, an amusing, low-budget British film about the adventures of a young man caught between the allure of punk and disco.

What all of these works share, apart from the year of their setting, is a dramatic understanding of the transitional nature of 1977. For it was the year, too, when the postwar Butskellite consensus finally began to unravel, when the failure of the Heath administration, the high inflation of the mid-1970s and then the financial crises of the Callaghan years led the small group of maverick advisers and thinkers gathering around Margaret Thatcher and Keith Joseph to believe that history was at last moving in their direction. In the view of those seeking to mould a new right consensus, the country not only in decline, it was in thrall to an intellectually discredited “Keynesian” orthodoxy, in which equality and social justice were pursued at the expense of liberty; full employment at the expense of ruinously high inflation; and the collective - as represented by union power and big, interventionist government - was prized over the individual.

As Thatcher, once in power, began purging her opponents from the Cabinet, it was fashionable to be on the right again, and ideas that once seemed dangerously subversive acquired a radical chic. The left found itself in an alarmed retreat from which it took more than a generation to recover, but only through accepting many Thatcherite orthodoxies. The right had won the economic argument, if not the cultural one, too.

The early radical Conservatives (or Thatcherites, as they were later known) - many of whom, such as Alfred Sherman of the Centre for Policy Studies, were former Marxists - were propelled by messianic zeal. They wanted to remake not only the Conservative Party, but the entire nation. In 1977, Sherman and the Centre for Policy Studies published a series of pamphlets outlining the way ahead. The unions and the “overmanned” nationalised industries, in particular, were identified as being the enemies of progress in an enterprise economy. If Britain was to be modernised, so the argument went, then the industrial practices and policies of much of the postwar period would have to be reversed.

The feeling, as the political philosopher John Gray has written, was one of “radical discontinuity with the past”. The aim was not simply to return the Conservatives to power; it was to change society irreversibly. And there was something clandestine, almost Bolshevik, about the early Thatcherites: gathering in dining clubs, in think-tanks, or for monthly salons at Jonathan Aitken’s house in Lord North Street, Westminster, they saw themselves as fighting a war of ideas against an enfeebling postwar consensus. They saw themselves as mould-breakers, iconoclasts, even revolutionaries. They were romantics. And some of them were visionaries, too. Among those who attended Aitken’s salons were Richard Nixon, Ian MacGregor (who would later lead the Tory attack against Arthur Scargill and the miners), John Gray, John Aspinall, Roger Scruton, Enoch Powell, Peregrine Worsthorne, Paul Johnson and Tom Stoppard.

In 1977, I had recently left primary school, so I was largely unaware of the currents of change flowing beneath the surface of the country’s political life. What I did understand, however, growing up in the nowhere zone of the Essex-Hertfordshire borderlands, was that Britain in the mid- to late-1970s was an extraordinarily drab place in which to be young: closed, parochial, complacent, tired.

To recall the landscape of my early teenage years is to return to an unrecognizable England: a country that was racist and corrupt, and hostile to any kind of eccentricity or difference; one riven by strikes and social unrest. Britain was then a country renowned for little but its bad food, awful public architecture (one need only visit the “new” towns that were built after the war to realise how little thought went into the planning of our public spaces), police corruption, and industrial malaise. And reading the music press, the NME and other papers as I did then, I knew that others felt the same as I did, felt that the old order was rotten to the core and deserved to be blasted away. What else could explain the anger of the punks, and of the new-wave bands that followed them?

I worked as a paperboy throughout 1977. My main interest then was sport; but I still used to read the headlines on the front pages before pushing the papers through the various letterboxes (and sometimes into the eager mouth of a hostile dog) on my early morning rounds. I understood even then, just as the Queen prepared to celebrate her first 25 years on the throne, that all was not right in the country. For a start, young women kept being murdered. I shall never forget the strange sense of fear and excitement I felt whenever I arrived at the local “paper shop”, to be told by the newsagent - a severe, bald-headed former gold prospector - that “he’s struck again”. I always knew what he meant: that he was speaking about the Ripper. The Yorkshire Ripper. He seemed to haunt all of our imaginations in 1977.

I recall, on several occasions, returning home from school with friends to ring the police hotline, on which you could hear the “Ripper’s voice”. He had a Geordie accent and was taunting the officer in charge of investigations (the call was later revealed to be a hoax). David Peace, in his novel, captures well the atmosphere of unease and suspicion that surrounded the Ripper murders, which darkened still further when Peter Sutcliffe (who was not caught until January 1981) began, for the first time in 1977, to murder non-prostitutes. Peace is very good on the corruption of the West Yorkshire police and on evoking the stink and decay of the industrial landscape through which the Ripper moved. The newsagent, “Vic”, was one of Thatcher’s “little people”: a small, independent businessman who was insurgent on all fronts. He felt that the country, as he put it, had “gone to the dogs”, and never wearied of telling anyone willing to listen about who was to blame - Labour and socialism, needless to say.

Amusingly, Vic had an adolescent son, a giant man-boy, who played centre-half for the best youth football team in our town and of whom his father was inordinately proud (I later found out that the boy was adopted). Until, that was, he became a punk and began turning up for his paper round (which was much shorter than the rest of ours) with his hair spiked, wearing painted boots, ragged tartan trousers and a ripped leather jacket. This seemed to disappoint his father.

But he ought not to have been too hard on his man-boy, because father’s and son’s rebellions were rooted in the same mutual disgust with the mediocrity and complacency of 1970s Britain. In this respect, I think, punk can be seen as a right-wing, consensus-smashing and counter-revolutionary ideology. There were avowedly left-wing punk bands, such as The Clash. But, more often than not, the more interesting punk and new-wave bands - Sex Pistols, Siouxsie and the Banshees, Joy Division - experimented with fascistic imagery and, like the Thatcherites, were set on breaking violently with the past. There was a darkness to their vision that was resolutely anti-Utopian: they knew, like believers in original sin, that they inhabited the worst of all possible worlds.

Punk was, essentially, a movement of working-class disaffection. It was the music of rebellion and simplicity. Anyone could be a punk: you didn’t even have to be an accomplished musician, or have access to expensive recording equipment. You simply required attitude, a few battered instruments and a rage to be different. “Punk stripped rock music down to the basics, and that was fantastic,” said Tony Wilson, the founder of the influential, independent Factory Records and the Hacienda nightclub in Manchester.

Many of the punk bands, Sex Pistols included, experimented with nihilism; but Joy Division - whom Wilson managed and whose singer-songwriter, Ian Curtis, committed suicide in 1980 at the age of 23 - meant it. Joy Division were never strictly a punk band - even in 1977, when the members were novices and their music had a hard-edged, DIY rawness. There was a complexity and lyrical sophistication to their sound that was quite unlike any other band at the time. Bernard Sumner - the bass player in Joy Division, who later became the frontman of New Order - has spoken of how the melancholy of Joy Division’s sound was an expression of the hopelessness and monotony of the post-industrial landscape of the Manchester of his childhood. “In those days, when you left school, you got a dead-end job that you hated. You grafted, and it rained all the time. There was mass unemployment. The old factories were coming down. Unoccupied buildings, all the windows smashed in. It was virtually a ghost town. You left school and went: ‘Oh, God. This is it.’”

In 1977, many people must have left school and asked themselves the same thing - if this, indeed, was it. Certainly, the Labour government, dominated by the unions, and moving between opportunistic monetarism and disastrous mismanagement of the economy, offered little hope or inspiration. If Labour were left, and were this mediocre, shouldn’t we think about moving to the right? Many did - including a disaffected Ian Curtis, who voted Conservative at the 1979 election, as did Harold Pinter and many other self-styled radicals like him.

One of my most vivid memories of 1977 is of being taken by my parents to central London to watch the Queen’s Silver Jubilee parade. I recall - however inaccurately - a cool, grey day, and impatiently mingling with the crowds lining the route that her absurdly opulent carriage would take. I remember glimpsing the Queen as she passed, waving from her carriage, and my lingering feelings of perplexity and bewilderment. Despite cheering gratuitously, I felt no sense of joy; nor, I am sure, did my parents, who were not so much enthusiastic monarchists as curious to see what all the fuss was about, and eager to be part of it all. But of what exactly? In retrospect, the Queen’s journey through London seems more like a funeral parade than a celebration of a vibrant nation.

More enjoyable was our street party in Essex, the only public event of its kind I have ever attended. Meticulously planned, organised and funded by a residents’ committee, the event was a success, combining street games, theatre, fancy-dress competitions and a night-time disco held at a local school. The only source of discord during the entire day was when, at the disco, a group of lads attempted to bully the DJ into playing the Sex Pistols’ gloriously ironic “God Save the Queen”, the banned single that was, fittingly, the unofficial No 1 in Jubilee week. It began: “God save the Queen/Fascist regime/She makes you a moron . . .”

It scarcely needs saying that the Thatcherite counter-revolution curdled, as most revolutions do, into something unpleasant and rigidly dogmatic. And yet, who can deny, even those on the old left, that there was something thrilling about the way in which a small group of maverick, renegade thinkers were able, through the force of their ideas, to move from the margins of a major political party to win control of its very centre, and in so doing remake the nation?

In its own way, the punk counter-revolution, which enjoyed its apotheosis in 1977 when “God Save the Queen” reached No 1 in the charts, remade the rock business, too. After punk, 1,000 small bands and record companies bloomed, launching an entire “indie” scene. But the energy of the punk scene was, in truth, quickly dissipated once the leading bands became even remotely successful. Because punk, like Thatcherism, was a movement of opposition, contempt and feelings of disenfranchisement nurtured it. As Alfred Sherman has said of his early commitment to radical Conservatism (but his words could equally apply to what Malcolm McLaren did with the Pistols): “Someone had to question the unquestioned, think the unthinkable, set our problems within a broader historical framework.”

In 1977, as women were being murdered in Yorkshire and the Queen was celebrating with a uniquely British pomposity, the nation was falling apart and someone needed to think the unthinkable, politically and culturally. In retrospect, the Silver Jubilee served not as the start of something, but as the beginning of the end. It was the point at which Britain realised it could go forward only by first going backward to dismantle the structures of the entire postwar consensus. The years that followed were often difficult and painful. There were inner-city riots, mass unemployment and a long, dispiriting miners’ strike. But, for better or worse, a sense of continuity in British society had been broken. The country would never be the same again.