Adam Boulton: Why to expect a surprise in the Tory leadership contest

Loyal defenders of Boris Johnson argued repeatedly that he should not be deposed as Conservative leader because there was no obvious candidate to replace him.

In doing so they misunderstood their own party. More often than not, the imperative when the Tories change leader is to get rid of the present incumbent and then worry about who is going to take over.

Once the present leader has gone toxic, the Tories know they need to refresh their public image but they are not sure how they are going to do it until they have argued it out through a selection process.

There is a long list of leadership winners who were not the bookies’ favourites when the Conservative party’s selection processes began – including Alec Douglas Home, Margaret Thatcher, John Major, David Cameron and Theresa May. All became prime minister.

Douglas Home, Major and May got the keys to Number 10 automatically, without the general public having a say, since the Conservatives were in power. Thatcher and Cameron had to win a general election first.

The uncertainty principle about who comes next has worked throughout the various different rules the Tory Party has introduced for selecting its leader, designed to bring democracy and accountability first to MPs and then the party membership as well.

It is coming into play once again this summer as it is far from certain that the Establishment candidate, former Chancellor Rishi Sunak, will manage a clear route to victory over the hurdles of repeated rounds of voting by calculating MPs, followed by the ballot of party members.

Before 1965 there were no rules. The next leader “emerged” after “the traditional processes” of “soundings” of MPs by what was bitterly called “a magic circle” of grandees.

Still, in 1963 when Harold Macmillan stood down, the eventual winner, Lord Home of the Hersel, was not a declared candidate or even properly available since he was a member of the Lords.

After an informal beauty contest during the party conference season, the Tories lost confidence in the front runner Rab Butler and the reserve, Chancellor Reginald Maudling. Douglas Home’s foreign affairs speech had gone down well so he went through the cumbersome process – ironically pioneered by the left-wing Lord Stansgate, aka Tony Benn – of abandoning his hereditary peerage to become an MP.

The UK briefly had a prime minister who was a member of neither of the Houses of Parliament before the newly knighted “Sir” Alec won the Kinross and West Perthshire by-election.

Labour’s Harold Wilson won the general election in 1964. To modernise their image, the Conservatives introduced elections by Tory MPs only to choose the leader. The repeatedly revised rules making it easier to unseat the incumbent were just as important. In spite of losing office in two close elections in 1974, very few Conservatives – least of all the party leader Ted Heath – saw Margaret Thatcher coming.

When Conservative MPs felt Thatcher had outlived her usefulness more than a decade later, leadership challenges in successive years provided the mechanism to push her out.

Michael Heseltine had rallied the internal opposition to Thatcher, but neither he nor veteran cabinet ministers such as Douglas Hurd won the leadership.

That job, and with it the prime ministership, went instead to John Major, who shot like a meteor out of political obscurity.

Major pulled off a shock general election victory in 1992 but went down to a landslide defeat at the hands of Labour’s Tony Blair in 1997.

The Conservatives set about refreshing themselves in opposition. The veteran cabinet minister Ken Clarke had the most votes in the first two rounds of voting by Conservative MPs. But the relatively inexperienced rising star William Hague eventually won through as other candidates were eliminated.

Hague tried to boost party membership by changing the rules for electing the leader. From then on, a vote by ordinary subscription-paying party members would make the final choice from a two-person shortlist assembled by MPs.

This introduced further uncertainty as to who would emerge as winner, since the careful calculations of MPs might not mesh with the emotions of the less equivocal party members.

Sure enough, the 2001 Tory leadership contest after Blair’s second landslide victory was unpredictable from the start.

First the campaign of golden boy Michael Portillo exploded on the launchpad, when he was unable to improve on his first-place lead. Seasoned ministers David Davis and Michael Ancram also fell by the wayside. And when the voters were offered the choice between the perennial candidate Ken Clarke and the little-known right-winger Iain Duncan Smith, they took a punt on IDS.

That went well. Two years later he was sacked in a vote of no confidence by his own MPs.

The parade of unpredictable winners continued. David Cameron came from behind to trounce David Davis. After Cameron lost the EU referendum and quit in 2016, Boris Johnson, the face of Brexit, missed his chance, so did Andrea Leadsom another Brexiteer. The Tories ended up being led by Theresa May who had been a Remainer.

Meanwhile Labour underwent its own leadership traumas.

The weighted electoral college of trade unions delivered Ed Miliband over his brother David, the preferred candidate of the party’s MPs and membership.

Next, after Ed tried to democratise the system by introducing one member one vote, a rush of new sign-ups delivered the far-left Jeremy Corbyn as his successor.

Occasionally, as happened with Michael Howard after Duncan Smith’s time as leader of the opposition and when Mrs May was the only candidate left standing, the party leader has been anointed without the need of a vote by the grassroots. But this infuriates activists. In any case, this year’s leadership is already hotly contested.

Only Boris Johnston has raised the possibility of a leader emerging by acclamation this month at Westminster. Six weeks have already been set aside for hustings by the last two candidates, a postal vote by members, followed by the declaration of the winner on 5 September.

Rishi Sunak is the Establishment candidate and he led the first round of voting by MPs. But his 88 votes were less than those of first-round leaders who have gone on to win the Tory leadership. In the second, round the independent Brexiteer Penny Mordaunt was besting Johnson loyalist Liz Truss.

There is still plenty of room for unforeseen outcomes in the most diverse field ever of candidates.

On past form of Tory election contests the real surprise will be if there isn’t a “surprise winner”.

Sky News is hosting a live TV debate with the contenders vying to be the next leader of the Conservative Party and therefore prime minister, and you could be in the virtual audience.

The debate will take place on Monday 18 July hosted by Sky News presenter Kay Burley.

If you would like to be a member of the virtual audience and be in with a chance of asking a question, please email NewsDebates@sky.uk.