As a beaming Rishi Sunak appeared for a fleeting early morning media clip in Ruislip’s Rumbling Tum cafe in west London on Friday, anyone tucking into their fry-ups who was unaware of the results of last week’s three byelections could have been forgiven for thinking that the prime minister had secured a huge breakthrough. “The Labour party has been acting like it’s a done deal – the people of Uxbridge just told all of them that it’s not,” he told the film crew in his brief visit. “When confronted with the actual reality of the Labour party, when there’s an actual choice on a matter of substance at stake, people vote Conservative.” It was a bold claim for a leader who had just seen his party suffer an average swing against it of a whopping 21.4% across the byelections, on top of seeing the majority in Boris Johnson’s old Uxbridge and South Ruislip seat slashed from more than 7,000 at the last election to less than 500 by Friday morning. Yet against the distorting prism of Westminster’s expectation game, the surprise Tory defence was the chink of light that Sunak needed to obscure the bleak raw data. And within the unusually idiosyncratic Uxbridge contest, he and his advisers scoured for the secret ingredient that could be used to reverse his party’s fortunes nationwide in the months ahead. About 200 miles away in the Selby and Ainsty seat, Keir Starmer was soon experiencing an equally confounding outing. When the Labour leader turned up at the ground of Selby Town FC on Friday morning to celebrate the party’s victory in the North Yorkshire seat, he wanted the message to be about a historic breakthrough for Labour in the Tory heartlands. Overturning a 20,000 Conservative majority had never been achieved by Labour before in any byelection. Starmer told the invited audience, which contained several members of the shadow cabinet and dozens of local party members, how delighted he was to be in “Labour Selby and Ainsty,” adding: “Doesn’t that sound good?” Yet when the questions came, they skated over Labour’s win to another issue less immediately comfortable: the reasons for the party’s disappointing failure to capture Uxbridge and make it three Tory losses in one night, after a thumping win by the Liberal Democrats in Somerton and Frome. The result had a tangible impact on Starmer’s event, which lacked the genuine excitement that would surely have been evident had everything gone Labour’s way the night before. Starmer found himself standing in North Yorkshire being quizzed about what should be done about the ultra low-emission zone (Ulez) that the Labour mayor of London, Sadiq Khan, is expanding around the capital, and which the Tories had exploited highly effectively as a big cost of living issue during the Uxbridge campaign. Starmer, mindful that there are more than 40 Labour MPs in Greater London who will now be worrying about the effect that Ulez may have on their votes at a general election, said the whole idea needed looking at again and that it was up to the mayor of London and Labour to review it, without being specific about what this may mean. In the immediate aftermath of the results, the sheer novelty of the Tory win in Uxbridge distracted from the ruling party’s evident struggles . The latest Opinium poll for the Observer gives Labour a 17-point lead heading into Westminster’s summer recess. It has 42% support, with the Tories down to just 25% of the vote. It is not hard to see why Labour officials were frustrated. Not only was the Ulez expansion a local issue affecting a cluster of seats around London, it is unlikely to still be the dominant problem at a general election. Meanwhile, the byelection took place – at the government’s choice – just weeks before the charge was set to be imposed. Ministers have also declined to provide any funding for a scrappage scheme to help those replacing old, polluting cars. Even senior Labour figures admit, however, that there were dangers in the Uxbridge result – not least because Starmer’s caution leaves some in his own team wondering what his positive offer to voters looks like. It was easy enough to find scepticism about Starmer and Labour in Uxbridge on Friday. “I don’t want Ulez, so I voted Conservative,” said Anna Stevens, 51, from South Ruislip. “I’ve got an older car, and in a couple of years’ time, they’ll probably change the age, and I won’t be able to drive it. My sister had to buy a new car and she only got a new car three years beforehand.” While only affecting a small minority of car owners, the Ulez policy had clearly taken on a significance beyond those with the old vehicles being targeted. Don Ackrell, 77, said the issue had already been the talk of his church group. “It doesn’t affect me because I get a bus, but the thing is that it’s got to affect all businesses around – it’s got to,” he said. While Ulez was mentioned by many voters who spoke to the Observer, it was often accompanied by concerns about Starmer. “I don’t know where he’s going,” said Ackrell. “I think he sits on the fence too much. I can’t see what Labour will do differently.” Such is the obsession among Starmer’s allies to ditch anything that may prevent Labour seizing power, one senior party figure said there had to be a response from the party. “City Hall is going to have to think about this,” they said. “It would be disrespectful to voters not to. People like the idea of green policies until they stop them doing something or cost them money. We have to have an approach that helps people with this stuff. We need to reassure people. I don’t want anything that will get in the way of a Labour victory.” By Saturday, Starmer himself was explicit about the dangers of the Uxbridge result as he addressed a large Labour policy forum in Nottingham. “That result in Uxbridge demonstrates there is never any reason to be complacent and never a reason to rest on our laurels,” he said. “We are doing something very wrong if policies put forward by the Labour party end up on each and every Tory leaflet. We’ve got to face up to that and learn the lessons.” But has a week of such relatively small gains really filled Sunak with confidence, or shown him a credible route to retaining power? For many of the mainstream Tory MPs in his team, there seems to be a genuine belief, enhanced by the Uxbridge result, that Labour is fallible and beatable – but they are also aware of the political mountain they face. “It’s not like the party can’t read opinion polls,” concedes one minister. “Everyone knows that future prospects depend on what happens with the economy, plus tangible action on NHS waiting lists and small boats. Plus, the party not giving up and falling into infighting. “I think Selby shows the government is seriously unpopular at the moment, but Uxbridge shows all hope is not lost. It’s also a reminder of Labour’s uncanny ability to snatch defeat from the jaws of victory.” Another former minister believes that in the Uxbridge victory, they have seen the beginnings of a strategy to beat Labour. “It shows that, where the voters know what a Labour administration looks like, they struggle to support Labour,” they said. “We would be nuts to try to spin this as anything other than very worrying. But Starmer is not winning. He is no Tony Blair. All is not lost and Sunak is a good, honest leader. I remain a realistic optimist.” The idea of using Ulez and other local examples – such as Labour’s record in government in Wales – to claim to voters that painful things happen when Labour is in power is most likely to become a new and prominent attack line in Downing Street’s recovery plan. Sunak is also said to be planning to become more sharply political in the months ahead. Yet the Uxbridge result has also had the effect of emboldening his internal enemies. For the right, Uxbridge was proof that the Tories prosper when they put clear blue water between themselves and Labour. It was, they argue, another reason for tax cuts as soon as possible, as well as ditching green policies. “There was a clear issue between us,” said a former cabinet minister. “People decided that they preferred our stance. People are displeased with us because they voted Conservative, expecting Conservative policies to be implemented. And a lot of people, whether rightly or wrongly, think that we’re not doing that at the moment.” Another hint of Sunak’s upbeat demeanour recently has been his willingness to make light of the Tory predicament. He has been telling friends that he recently saw the new Tom Cruise film, Mission Impossible: Dead Reckoning Part One – inviting comparisons with his party’s plight. The question for many of his MPs is whether he really is drawing up the script to overhaul Labour, or whether he is in danger of confusing fact with a more palatable fiction.