Tory MPs critical of Rishi Sunak’s leadership are hoping he will stand down voluntarily to avoid the spectacle of a damaging coup and are looking to May’s local elections as a potential crunch point, the Guardian has been told. A former minister said several Conservative MPs had contacted Graham Brady, who heads the 1922 Committee of backbench Tories, to say they want the prime minister to quit, but that they had not sent in letters of no confidence yet. The former minister said they believed Sunak’s Downing Street was too complacent about the risk he faces after no new public challenges emerged after two heavy byelection defeats for the party this week. “No 10 think they’re not in much trouble at all over a [leadership] challenge, which feels a bit arrogant,” they said. “Quite a lot of moderate MPs now agree with the traditional anti-Rishi types to say that the best course of action is not for us to remove him, but for him to stand down voluntarily, and they’re speaking to Graham Brady about it. That’s why you’re not hearing much noise about it. “I know it’s happening. The magnitude of it is difficult to say. It avoids the pain of removing a PM and might even prevent a protracted leadership contest, if one person emerges as a successor, like Rishi did.” The situation was less like the removal of Boris Johnson, they said, than “a slower-motion and less contentious Liz Truss”. “It’s not that they think Rishi is terrible, like she was. But he’s had a go for 18 months and the plan isn’t working,” they said. Since Labour won the Kingswood and Wellingborough seats, overturning sizeable Conservative majorities, Sunak has faced renewed calls from some of his MPs to tack further to the right on areas such as immigration and tax cuts, especially after Reform UK, formed by Nigel Farage, won 10%-plus of the vote in both contests. As yet no MPs have joined Simon Clarke and Andrea Jenkyns in calling publicly for him to go, but one Conservative source said many rebels had decided to wait and see how the party performed in local elections on 2 May. “This will be the crunch moment for a lot of those groups. That’s stick or twist time,” they said. “If the locals go badly, I would very much expect someone to launch a serious attempt to remove the PM. If Rishi is still PM on 31 May, he will be PM at the election.” Other Conservative MPs, however, have dismissed such ideas as unlikely and counterproductive. One long-serving backbencher said: “Never say never in politics, but I haven’t detected any signs from colleagues that we need to change the leader. Yes, some disagree on policy, but most people recognise that changing leaders literally months from an election is not a particularly good strategy. “I would quietly say to anyone unhappy with Rishi: he’s going to be going sooner or later anyway, most likely to the board of an investment bank, so you might as well wait for an election defeat.” Another former cabinet minister said: “One thing one always hear from voters is that they don’t like divided parties, so if anyone thinks the byelections mean we need to change leader, they’re taking the completely wrong lesson from them. “Also, when we have won elections, we have done so from the centre-right. So arguing that the answer is a leader who moves further to the right is also missing the point.” A government source said Sunak had achieved in a number of areas, including on tax cuts and migration: “His plan is a Conservative plan, it’s starting to work and only by sticking with it and uniting behind it can we take the fight to Labour.”