Indigenous communities in Bolivia have objected to Colombia’s plans to recover the remains of an 18th-century galleon believed to be carrying gold, silver and emeralds worth billions, calling on Spain and Unesco to step in and halt the project. Colombia hopes to begin recovering artefacts from the wreck of the San José in the coming months but the Caranga, Chicha and Killaka peoples in Bolivia propose that the galleon and its contents should be considered “common and shared patrimony”. A substantial part of the treasure onboard the San José is believed to have been mined by the forced labour of Indigenous peoples in Bolivia, so Colombia’s plans to lift the remains without consulting their descendants would violate international law, the communities said in a letter to Unesco this week. “Not having our consent, our participation and without taking into account how it will impact the present and future of our communities is irresponsible and contrary to justice,” they wrote. “We do not have the right to forget, and nor do Spain or any of the American republics … have the right to erase or change our memory.” The San José was carrying an immense bounty of gold, silver and emeralds from Latin America back to Spain in 1708 when it was sunk by a British fleet off the coast of Cartagena. Since the wreck was located 600 metres (nearly 2,000ft) underwater in 2015, the discovery has been mired in international legal disputes including an ongoing case in The Hague. Colombia, Spain and a US salvage company all lay claim to the wreck – which has been dubbed the “holy grail of shipwrecks” –and its cargo, thought to be worth as much as $17bn (£13bn). The Colombian government announced in February that it would soon begin exploring the shipwreck and eventually hopes to build a museum dedicated to the 150ft (46-metre) vessel. The government has put aside $7.3m for the first exploration phase which will require hi-tech robots to scour the seabed. But the Bolivian Indigenous communities argue that Colombia does not have the right to explore the San José without including the descendants of those who mined the precious metals which went into the treasures onboard. Much of the San José’s cargo is believed to have been dug up from the Potosí mines in southern Bolivia, making it “shared historical and cultural heritage”, they said. The Bolivian communities have requested that Spain and Unesco intervene before Colombia can begin unilaterally lifting the remains of the wreck from the seabed. “The memory of our people is attached to those remains that rest on those sunken ships, in a way that we do not choose, as one of the peoples who gave their work and their own history to work in the mines of Cerro Rico de Potosí,” the letter reads. The Potosí mines were the largest in the world and an economic engine of the Spanish empire. Historians believe that before the mountains were stripped bare of the precious metal they were the source of more than half of the world’s silver. The riches were dug up by Indigenous people and African slaves then carried on llamas to the coast and transported to Europe on Spanish treasure fleets such as the San José to fund colonial wars. “We think that extracting important fragments from that era from the bottom of the ocean is like discovering an island, and on that island live our memories and the objects that the work of our ancestors created,” the three Indigenous Bolivian groups said. The director of the Colombian Institute of Anthropology and History, Alhena Caicedo, recently told the Guardian that, in accordance with Unesco’s requests, the country would not sell any precious artefacts recovered. The exclusion of Indigenous communities in the discussion of the fate of the San José was “inexplicable”, said José María Lancho, a lawyer in Madrid representing the three Bolivian groups. “We have enough rights, enough historical experience and I want to believe enough morals to be able to accept and accommodate the claims of those communities …intervening in the site without the consent of these people would be a form of plunder,” he told the Guardian. Spain and Colombia were missing out on a “a historic opportunity” for reconciliation, he added. Should the Colombian government lift any elements from the San José and use them for profit, even if they are placed in a museum, Indigenous communities should be paid “fair compensation for the use of our cultural legacy”, the Bolivian communities argued in their letter. The Colombian ministry for culture did not comment on the letter but said that it had invited another Indigenous community, the Qhara Qhara, to participate in its plans to explore the ship’s remains. While the Caranga, Chicha and Killaka lay claim to the San José’s treasure on the grounds that their ancestors were forced to dig up and process the silver, the Qhara Qhara lay claim to the archaeological treasures through their rights to the land.