As we sink deeper into the digital age, a problem emerges. People dislike the idea of giving up their data to some vast and powerful entity. But, increasingly, you can’t run things without them. The dilemma sharpens when it comes to health systems. These involve the most sensitive kind of personal information – the sort you’d really rather not end up in the hands of insurers, say, or a future employer. Yet without a powerful data system to crunch through everyone’s medical histories, things fall apart. The right file from your GP doesn’t quite get to the paramedics in time and they give you the wrong treatment. You move from one city to another and somehow half your vaccine record goes missing. At present, the NHS’s database lags behind those of other rich countries. Health data is deposited in pockets, geographically – some with your GP, and some across hospitals you may have attended. No one gets a complete picture. Surgeons must sometimes rely on patients to remember and recite their health records: a risk. Meanwhile, without a competent data system to direct demand to supply, much-needed beds and operating theatres lie empty. Other, bigger opportunities are also lying fallow. NHS data is unique – for decades, doctors have been required to code records in machine-readable form. This is a hugely rich data set: a gift to medical research. Who knows what new treatments might emerge if we gathered up all the files languishing at the back of cabinets and stuck them all on a database? Governments have been trying to stitch together our patchwork system for decades. Billions have been lost in these attempts. But they always run up against the same problem: people just don’t want to share their medical data, even when assured it will be anonymised. When the government aimed to build a collection of anonymous GP health records, around a million patients opted out. “Certain demographic groups were highly represented in that group – for example, women were more likely to opt out than men,” said Simon Bolton, who until this January was chief executive of NHS Digital. “That means any research on the remaining data could not be as effective.” The latest of these attempts has closed a loophole: patients cannot now opt out. But this has enraged civil liberties groups, which are concerned about the company chosen to merge, clean and provide tools for sorting through the data. This is Palantir, a Silicon Valley data analysis firm, named after elf-made crystal balls in The Lord of the Rings novels (stones that are only reliable in the hands of those who “possess great strength of will and of mind”). Its previous clients have included the CIA and FBI; it has been involved in predictive policing; and its co-founder Peter Thiel has made remarks suggesting Brits have “Stockholm syndrome” when it comes to the NHS and the best thing to do would be to “rip the whole thing from the ground and start over”. This is all terrible PR for the project, and pro-privacy groups are right to want assurances this data will be handled properly – after all, there is only so far you can anonymise health data, which gets specific pretty fast. Campaigners worry it is possible for Palantir to sell on your information. But they should consider two reassuring facts. First, the law stands in the way. Palantir does not have legal access to this data, and should it break the law, the information commissioner’s office has the power to sue it into oblivion. And second, the company lacks incentives for this type of bad behaviour. It is not seeking to own the information encoded in the datasets. It is not in the business of analysing or selling information, but of providing software for handling highly sensitive data, such as for the CIA. Lose that reputation and it loses everything. They should also consider that there is a trade-off here. Patients are already suffering for want of a joined-up system. The NHS was always going to need help from the private sector. “It barely has capacity to get through waiting lists, let alone run a massive data centre,” says Lynette Nusbacher, chairwoman of KryptoKloud, a cloud security business, and former senior intelligence adviser to the cabinet office. “We are going to be drowned in an unmanageable sea of data.” Drumming up public concern over the project makes it more likely to fail, which carries its own risks: lost years, lost treatments, lost lives. We should be careful about encouraging digital nimbyism when so much is at stake. Then, too, loud laments about the likes of Thiel – Palantir has distanced itself from his remarks – may provide cover for more worrying problems. Cori Crider, co-founder of Foxglove, a group campaigning against Palantir and the proposed data platform, has raised the fact that the Home Office has a history of pushing to obtain health data on illegal migrants. This is indeed a matter of concern – but with the government, not the company contracted to provide the plumbing around the dataset. The government, remember, already has access to our medical data. It is the government that ultimately decides what is done with it, and which has some incentives, as in the case of migrants, to misuse it. Elena Simperl, professor of computer science at King’s College London, said that the NHS has yet to publish a complete list of the uses to which this data is to be put. For Palantir, the real problem is whether it can deliver. As with any big IT contract, there is a risk the NHS gets locked into this particular software, making it hard to extricate itself if something better comes along. And it may not be up to this huge task. “It might be better to start small and prove the solution at a small scale and then expand,” says Bolton. “This is a single deal with one consortium. There’s quite a lot of risk that it goes wrong.” Martha Gill is an Observer columnist