Rishi Sunak has now survived 100 days as prime minister – a pretty small feat by historical standards but twice the length of his predecessor’s term. But the circumstances he inherited – the lack of mandate, plummeting polls and an economy in freefall – have deeply constrained what he is able to do with his time in office. Inside No 10, key advisers argue that Sunak has already made significant progress, that the economy is starting to recover, that he has set out a clear stall with five pledges on inflation, national debt, the economy, immigration and the NHS, and that he has largely taken the Tory party infighting off the front pages. But he is on the brink of losing his third cabinet minister in ignoble circumstances, amid a bullying investigation into one of his closest allies, Dominic Raab. He has already lost Gavin Williamson and Nadhim Zahawi, both to scandals. He has already caved to three rebellions, on planning, onshore wind and the online harms bill. And the polls have not even begun to turn, with Labour enjoying a healthy 20-point lead and Sunak’s own ratings beginning to sink. “We are under no illusion – there are many traps ahead,” one Downing Street source said. “We have Northern Ireland, the budget – both times when we may have to find a way to bring colleagues with us when they might not be always be inclined to.” Sunak’s No 10 is a much more functional environment than it was under the last two incumbents, though some say it has been a challenge for him to adapt to the role. Sunak as chancellor had what one aide described as a “party trick” of being able to recall details from the subsections of detailed reports, which he could test officials with by page number. As PM, he has had to home in on priorities – but some say he is struggling to shake off micromanagement. One person present at a recent meeting of NHS leaders said they were baffled why Sunak was grilling them on the intricate detail of how a particular IT system worked. Another minister observed: “I think at the moment he is still trying to be chancellor, home secretary and prime minister.” But Downing Street sources say this is not true. “He is self-aware – he knows you cannot do that and he is happy to delegate. That’s why you have seen him pick the five priorities.” After Liz Truss left office, polls suggested that voters wanted to keep an open mind about Sunak and rated him significantly higher than his party. That is now beginning to turn. According to senior Labour figures, their most recent focus groups, with swing voters in Southampton, Dewsbury and Bury last week, were described as being “utterly brutal for Sunak”, with participants engaging in “open mockery” of the prime minister. Even the most pessimistic members of Keir Starmer’s team say they have seen a decisive shift. Voters were scathing about attempts to tackle the cost of living and suggested they believed fundamental public services like the NHS were broken. From the focus groups, there was a growing feeling many had not seen much of Sunak since coming to power – though Sunak is trying to shift this narrative with a series of “PM Connect” events across the country, spending well over his allotted time speaking directly to local voters. But the most common refrain from voters was the one that Tory MPs are also the most nervous about – that Sunak is “out of touch” and that his wealth means he cannot understand voters’ concerns. One voter in Bury told Labour’s researchers that they could not take Sunak seriously when he spoke about the NHS, because it was obvious he had never been on a waiting list. Sunak gathered his cabinet for an away day at Chequers last week, where the Tory election guru Isaac Levido is said to have told ministers that there was a narrow path to victory because of the number of voters still undecided about Starmer – approximately 20% of the electorate. But some Tory MPs who have been in discussions with CCHQ about electoral strategy for their seats say some of that briefing paints far too positive a picture compared with what they have seen on the ground. One said they had been warned that those with a majority of less than 10,000 were at risk of losing seats. Prof Tim Bale, of Queen Mary University, who has authored several books on the Conservatives, said polling currently suggested people “no longer believe the Conservatives have the solutions to everyday problems”. He said there was very little to differentiate Sunak at present from the public perception of the Conservatives. “Really, it just looks like continuity Conservative government,” he said. “I think generally speaking people do recognise now, after Johnson and all the plotting, that competence is not too bad a thing. But they could also get that from Keir Starmer,” he said. The places where the Conservatives are likely to face the most difficulty are the north of England and the Midlands, and there is frustration among the new intake that Sunak has almost no one from the so-called “red wall” at any kind of senior level in the government. “I think Rishi actually sees himself as being the northern representative around the table, which I don’t think is helping,” one minister said. That may change with a rumoured summer reshuffle, though even some of his supporters who were not rewarded with jobs initially say it has been a good idea to keep a number of Truss supporters inside the tent, such as Thérèse Coffey, the environment secretary, Steve Baker, a Northern Ireland minister, or the communities minister Dehenna Davidson. It was once rumoured that Sunak intended to eventually replace Hunt, but senior allies now say that is not the case. “They get on very well – I think if he had been minded to move Hunt at a summer reshuffle, that won’t happen now,” one key backer said. And Suella Braverman, seen as a scandal waiting to happen by Labour and not Sunak’s natural ally, is also privately viewed more sympathetically. One senior No 10 source said: “She is better than many people think, I have been genuinely impressed.” MPs are split into a number of camps regarding how to deal with the polling predicament. There are those who believe the election can still be narrowly won, though many of them, including those close to Sunak, believe he needs to be bolder on drawing new dividing lines – including on immigration. Several of them, including a number of ministers, are pushing for Sunak to commit to leaving the European court of human rights as a manifesto pledge. “I think it’s an issue where you will get most red wall MPs onside, many other colleagues can be convinced of that course and the few who can’t are not really so important,” one minister said. Another small rump of MPs, including several in the new Conservative Growth Group, are convinced that the resurrection of Johnson is the only remedy, but most will concede that cannot happen until early 2024. But the majority are focused on working hard locally to try to rescue their own seats, including taking part in rebellions such as those on housebuilding, to send a signal to their electorate. Those who do not need to – mostly younger and still ambitious MPs – are already meeting to think about what comes after 2024, to start building alliances and power bases within the party to determine who will triumph out of the postmortem.