Lindsay Hoyle has come out fighting against attempts to oust him as Commons speaker as Rishi Sunak criticised his “very concerning” choice over the Gaza debate. Hoyle is under pressure after 67 Conservative and SNP MPs signed a motion calling for a vote of no confidence in him. The furore erupted on Wednesday when Hoyle changed parliamentary procedure to allow a Labour amendment on Gaza to be debated to help stave off threats to MPs, sidelining the SNP’s original motion. Sunak said on Thursday that Hoyle had changed the “usual ways in which parliament works”, which he said was “very concerning”, arguing that parliamentarians should never be intimidated by “extremists”. He also said the speaker had “apologised for that and is going to reflect on what happened”, indicating that he appeared to be willing to draw a line under the episode for now, while government sources said they were not backing demands for Hoyle’s removal. Sunak said: “I think the important point here is that we should never let extremists intimidate us into changing the way in which parliament works. Parliament is an important place for us to have these debates. And just because some people may want to stifle that with intimidation or aggressive behaviour, we should not bend to that and change how parliament works. That’s a very slippery slope.” By Thursday evening momentum against the speaker appeared to be slowing. The Foreign Office minister Andrew Mitchell, who led for the government during the Gaza debate, told Times Radio: “If you ask me whether I am calling for his head, the answer is I’m not.” He added: “The speaker must speak for himself, and as I said, I think it was a controversial thing to do. He has apologised for it. As far as I’m concerned, I’m prepared to leave it at that.” Earlier, Sunak’s spokesperson declined several times to express confidence in Hoyle, but the threat to the speaker’s future appeared to diminish as no government figures called for his resignation. Hoyle on Thursday again apologised for a “wrong decision” and said his mistake had been made in pursuit of looking after MPs, after being told of “absolutely frightening” threats against them if the Labour position was not debated. The SNP originally tabled a motion for debate calling for an immediate ceasefire in Gaza, while Labour’s motion argued for the same with some conditions about pushing for a two-state solution and working with international partners. A Conservative motion also called for a ceasefire with an even longer list of conditions. Hoyle’s decision to allow the Labour amendment led the government to withdraw its own motion on Gaza. The Labour motion subsequently passed, with Tory sources confirming that it was partly because they did not have enough time or votes to get their own amendment through the Commons. Hoyle said the SNP could have an emergency debate on Gaza, but the offer failed to placate its Westminster leader, Stephen Flynn, who said the speaker’s position was not tenable. Penny Mordaunt, the leader of the Commons, defended Hoyle as a “decent man”. But while the government is not pushing for his removal, Kemi Badenoch, the business secretary, retweeted a scathing intervention from the former attorney general Geoffrey Cox saying Hoyle’s reasoning was unacceptable. “There are two possible explanations for the speaker’s decision to abandon longstanding convention,” Cox said. “First, he did it to assist his former party leader [to] get out of a bind. Secondly, as he says, he did it in a misguided attempt to protect certain Labour MPs from the intimidation they said would otherwise have followed if they had voted against the SNP motion. “Either reason is unacceptable. If the former, it is an abuse of his office. If the latter, it is an abject surrender to intolerance and tyranny; it meekly offers up the House of Commons as able to be influenced by external threats.” During an occasionally tense Commons session on Thursday morning, opinions were split over Hoyle’s fate. Addressing the speaker directly, Flynn said: “As I have expressed to you privately prior to proceedings here today, we do not on these benches believe that you can continue in your role as speaker if we do not have confidence in your ability to do so.” The speaker received words of support from many MPs, including on the Conservative benches. Sir Edward Leigh, a Tory backbencher, said: “I think we should move on now and I would recommend that we don’t put in motions of no competence.” Mark Francois, a veteran of the Tory right, spoke emotionally in Hoyle’s defence. Referring to the murder of his friend and fellow MP David Amess, Francois said: “I will remember everything that the speaker did to help me and all of us when our great friend, my best friend, was murdered … Mr Speaker went the extra mile for all of us to help us all deal with that tragedy.” Keir Starmer defended his decision to lobby Hoyle to allow the Labour amendment and rejected the idea that he had put pressure on him. Starmer met Hoyle hours before the vote, arguing that the speaker should ignore precedent. On Thursday Starmer denied allegations of threatening the speaker, telling journalists on a visit in Sussex: “I can categorically tell you that I did not threaten the speaker in any way whatsoever. I simply urged to ensure that we have the broadest possible debate. So that actually the most important thing, which is what do we do about the situation in Gaza, could be properly discussed by MPs with a number of options in front of them.”