Rishi Sunak struggled to keep control of his fractured party on a chaotic fifth day of the Tory election campaign, as one MP defected to Reform and a minister criticised the prime minister’s pledge to bring back national service. Sunak was in Buckinghamshire as he sought to get back on the front foot after a bruising start to the snap election, with Tory insiders increasingly worried about his strategy and performance. Sunak found himself under fire early on Monday, as Steve Baker, a Northern Ireland minister, said introducing mandatory national service was a policy dreamed up by advisers and sprung on candidates. It later emerged that Baker, who is defending the Labour target of Wycombe, had chosen to go on holiday to Greece rather than stay on the campaign trail – after Sunak previously told MPs they should go ahead and book time off. The prime minister was then hit with the defection of Lucy Allan, the Conservative MP for Telford, who said she would support the local candidate for Reform. The party suspended the whip, but she hit back saying she had quit first and that the Conservatives had no chance in her seat, according to the Shropshire Star. Sunak’s woes increased further when Zac Goldsmith, the Conservative peer and former minister who resigned over his unhappiness with Sunak’s lack of commitment to the environment, accused him of having “damaged the party almost beyond repair”. “The hope is that when Sunak disappears off to California in a few weeks there are at least some decent MPs left around which to rebuild,” Goldsmith said. Appearing in Amersham, where the Tories lost a byelection to the Lib Dems last year, Sunak ignored questions about the defection and criticism from Baker. He did, however, angrily respond to Goldsmith for predicting he would move to the US if the Conservatives lost the election, pledging to stay on and serve a full term as an MP. “I’m surprised at Lord Goldsmith, who I don’t think I’ve spoken to in a very long time, has some intimate knowledge of my family’s arrangements,” Sunak said. “Of course not. My kids are at school, this is my home, and as I said earlier, my football team just got promoted to the Premier League.” Sunak later told ITV that he would stay in the UK “for years” to come. The prime minister also struggled to say how his national service policy would work, refusing to clarify what fines or incentives would make it mandatory for 18-year-olds. One minister claimed that it could involve parents being fined if their grownup children did not comply, before another party official appeared to rule that out. The former defence secretary Ben Wallace defended the policy by appearing to suggest young people had few demands on them. “Heaven forbid young people are made to do something …,” he said. Jonathan Ashworth, a Labour shadow cabinet minister, said the policy was “continuing to unravel” and that the Tories could not just “shrug their shoulders” about the detail of what young people would be asked to do. “That will not do. Rishi Sunak cannot announce the flagship policy of his general election manifesto but then refuse to answer the most basic questions about how it will work, and what it will cost,” he said. The Labour leader, Keir Starmer, called the proposal “desperate”, claiming on Monday that the Conservatives were “rummaging around in the toy box to find any plan they can throw on the table”. He said: “I don’t think it’ll work – you’ve seen what military experts, those with experience have said about it, you’ve seen what the government said about it just a few days ago when they were asked.” The national service plan is part of a Conservative attempt to win back former Tory voters from 2019 who are now tempted by Reform. The prime minister announced on Monday night that the party would give a new £100-a-year tax break to pensioners by increasing their personal allowance, as part of his continuing drive to win back older former Conservative voters who are saying they intend to vote for Reform. Billed the “triple lock plus”, the policy is estimated to cost £2.4bn a year by 2029/30, a price tag that mirrors Sunak’s proposed national service duty, billed at £2.5bn a year. The tax cut will be funded through a previously announced plan to raise £6bn a year by the end of the next parliament by clamping down on tax avoidance and evasion. Some Conservative MPs, however, told the Guardian they were worried that the focus on potential Reform voters would sacrifice a lot of home counties seats to the Lib Dems, including ones such as Surrey Heath, where Michael Gove announced that he would not be standing again on Friday. Other Tory insiders are concerned about the wider strategy, and suggested a rethink of Sunak’s top team may be needed already, less than a week into the campaign. The prime minister’s decision to announce an election in the rain, which drew headlines about him drowning, as well as his campaign stop in the Titanic Quarter in Belfast, where he was asked whether he was captaining a sinking ship, have drawn criticism for making him appear hapless. “The base problem is, they had some things ready, like the framing and the moment they were looking for. But the operations side of things has not really been ready, the media and messaging has been all over the place. They kept it so tight, but maybe not everyone was in the room who needed to be. They just weren’t quite ready enough for this,” one insider said. Another strategist pointed the finger at the election guru Isaac Levido for allowing mistakes to occur, and “a narrative of the campaign being in crisis” despite briefings that the adviser had been among those pushing for an autumn election rather than a summer one. An internal Tory memo leaked to the Times showed party staff complaining about a lack of MPs and candidates getting behind the campaigning effort. Others defended the Conservative effort. Johnny Mercer, a Tory minister and MP for Plymouth, said Allan’s defection was “not good news, but it doesn’t mean the wheels are coming off the campaign”. He told Times Radio: “I think any campaign is difficult. I think that any political party is a broad church and you’re going to have people within that organisation at that time unhappy with aspects of what’s going on. That’s completely normal. “It’s disappointing. I like Lucy, but ultimately it is true that if you vote for Reform, you’re going to see Keir Starmer in Downing Street, and that doesn’t have any sort of intellectual sense about it if you want to vote for any of the Reform policies. So a vote for Reform is a vote for Keir Starmer.”