DHAKA, Aug 15 (Reuters) - More than two dozen Rohingya refugees were missing and feared drowned after their boat sank in the Bay of Bengal while trying to flee a remote Bangladeshi island, officials said on Sunday. "We are devastated that reportedly many passengers, including women and children, have tragically drowned. The confirmed number is not yet known," the United Nations refugee agency said. Despite criticism from rights groups, Bangladesh has moved nearly 20,000 Rohingya to Bhasan Char, which emerged from the sea only two decade ago and is considered vulnerable to floods. "Living here is not easy. There is no work. People can"t go and meet with their relatives in the camps in Cox"s Bazar," Abdul Hamid, 40, a Rohingya refugee in Bhasan Char, told Reuters by phone. "That"s why people are going crazy and taking risks to leave." The Rohingya, a Muslim minority group who fled violence in neighbouring Buddhist-majority Myanmar, are not allowed to move off the island, which is several hours" journey away from the southern port. Around 40 Rohingya refugees, including women and children, were in the fishing boat when it capsized due to bad weather early on Saturday, police official Rafiqul Islam said, adding that 14 people were rescued by fishermen and taken back to the island. The coast guard, navy and air force were continuing rescue operations, Islam said. Bangladesh wants to eventually transfer 100,000 Rohingya to the island to ease chronic overcrowding in its camps at Cox’s Bazar, home to more than 1 million Rohingya. Bangladesh says the relocation is voluntary, but some Rohingya have spoken of being coerced while several thousand angry at living conditions on the island protested in June during a visit by the United Nations team. "Dhaka should end its detention of refugees on Bhasan Char and grant freedom of movement for all people on the island," John Quinley, senior human rights specialist at the Asia-focussed charity Fortify Rights, told Reuters. "The reports of Rohingya dying while trying to flee the island by boat are extremely alarming," he said. "The government should allow any Rohingya who wish to return to Cox’s Bazar to do so, and facilitate the transportation." Police said scores of Rohingya refugee fled from the island, where refugees say they face restrictions on movement and a lack of means to earn a livelihood.