LONDON, Jan 16 (Reuters) - The British Conservative Party chairman rejected calls for Prime Minister Boris Johnson to resign but said he must address the culture within his government that resulted in multiple staff gatherings at his residence during coronavirus lockdowns. Johnson has apologised for attending a gathering in the garden of his Downing Street residence in May 2020 where staff had been invited to bring their own alcohol at a time when strict rules forbade the public from almost all socialising. Amid a public backlash at the perception that the government did not follow its own rules during the pandemic, an internal investigation is looking at that party and several others - including parties on the eve of Prince Philip"s funeral. read more "We need to find out the facts and then the Prime Minister needs to respond effectively and to address the culture in Downing Street," Conservative Party Chairman Oliver Dowden told Sky News on Sunday. The scandal has generated calls for Johnson to resign - including from within his own party - and seen the Conservatives fall far behind the opposition Labour Party in opinion polls. read more Opposition Labour Party leader Keir Starmer accused Johnson of breaking the law, and of lying to parliament when first challenged about the reports of parties. He said the public could no longer take the prime minister seriously when it came to the pandemic response. "I think he broke the law. I think he"s as good as admitted that he broke the law," Starmer told the BBC, citing Johnson"s apology to parliament for attending one such event. "I think he then lied about what had happened." Co-Chairman of British Conservative Party Oliver Dowden walks on Downing Street in London, Britain, December 17, 2021. REUTERS/Toby Melville Co-Chairman of British Conservative Party Oliver Dowden walks on Downing Street in London, Britain, December 17, 2021. REUTERS/Toby Melville INVESTIGATION Conservative Member of Parliament Tim Loughton joined the handful of lawmakers in Johnson"s party publicly calling for him to quit. "I have regretfully come to the conclusion that Boris Johnson’s position is now untenable, that his resignation is the only way to bring this whole unfortunate episode to an end," he wrote on Twitter on Saturday night. The investigation into the matter by senior civil servant Sue Gray would produce a factual account of events set against a summary of rules at the time, but would not advise on disciplinary actions, Dowden told the BBC. Her report would be published in full, he later told Times Radio. Facing the most serious threat to his leadership since he came to power in 2019, the Sunday Times reported Johnson would try to draw a line under the issue with a mass clear out of staff and a slew of populist policies. The Sunday Express reported that Johnson would remove all existing coronavirus restrictions at an upcoming review, including mandatory face masks and an instruction to work from home where possible. The report cited a government source. Asked about that, Dowden said: "I"m very hopeful and optimistic but clearly, we will await the data at the point of the decision before making that final decision." New cases of COVID-19 have begun falling after a sharp spike caused by the highly-transmissible but milder Omicron variant. Britain reported nearly 82,000 new cases on Saturday, less than half the peak seen in early January.