Iraqi farmer Shaker Salih says he feared ISIS, but feared its defeat even more. His problem is persuading people to believe that he did not support the extremists, said a Reuters report Thursday. When Iraqi forces drove ISIS from his home town in Salahuddin province, north of Baghdad, Salih left with the extremist group. He then stayed as long he could in its shrinking, self-declared “caliphate”. "We thought militias would kill us for living under ISIS, so we fled," said Salih, 49, referring to paramilitaries that helped defeat ISIS. "Thats why we stayed with ISIS. We were used to them and knew what to do to survive." He now lives in the sprawling, guarded al-Hol displacement camp across the border in Syria, where among 70,000 fellow Iraqis, Syrians and others, there are thousands of die-hard ISIS supporters. Some are passengers in his car, which he uses as a taxi to make a living by charging one dollar a trip. "If God allows, the caliphate will return," one Syrian woman at the camp, who gave her name as Fatima, told Reuters. Iraq is preparing to bring home its citizens from al-Hol, who number more than 30,000. But it is struggling to decide what to do with them - and how to identify those with genuine ISIS links and those simply caught up in its “caliphate”. The difficulty of distinguishing those groups, sometimes from the same clans and communities now in al-Hol, means that many like Salih face long-term detention under plans the government is weighing up. Baghdad recently abandoned the idea of building a separate internment camp for those coming from al-Hol after aid agencies on which it relies to support hundreds of thousands of displaced people opposed it, rights and aid workers say. The latest Iraqi proposal is to put them in buildings and more permanent structures in isolated areas guarded by security forces, according to officials, aid workers and rights groups. "Camps are temporary, people cant live there forever. The only solution is to designate areas monitored and protected by the state and provide services and work on reintegrating these people," said Ali Bayati, a member of Iraqs semi-official High Commission for Human Rights. Aid agencies have said they will not provide support for a new internment camp or detention zone, according to several aid workers, due to the risk of rights violations. They are seeking instead to put people who pass security screening into existing displacement camps, according to a plan seen by Reuters. Iraqs immigration and displacement ministry and the prime ministers office declined to comment on current plans for the al-Hol captives.Global issue Since most of those in al-Hol emerged from the last sliver of land held by ISIS in eastern Syria, it is not easy to sort those with extremist views from non-radicalized people and ensure the latter do not change their views. If Baghdad gets it wrong, global security could again be at risk, Western officials have said, as it was when ISIS - a reincarnation of al-Qaeda groups that had taken years to suppress - took hold in Sunni areas of Iraq among communities that felt persecuted by the Shiite-dominated government. Iraqis with clear links to ISIS, such as militants and their families, have mostly been detained and some of them have been transferred to Iraq to face trial. Many other residents in the camp, which is controlled by US-backed Syrian Kurdish forces, say they did not back ISIS but survived its brutal rule by keeping a low profile. Salih and other Iraqis from his region live separately from the suspected fighters in al-Hol, and call the foreigners, who are held in a different area, "extremists". When they return to Iraq, the government says keeping them away from existing camps and the general population will protect them from vigilante attacks by people who suffered under ISIS and help avoid the spread of extremist ideas. New York-based advocacy group Human Rights Watch (HRW) says al-Hol residents deemed a threat may be housed in abandoned or half-completed building projects or shipping containers converted into housing rather than in camps. "What they are now thinking of creating is more permanent. It means they foresee holding people there for longer," said Belkis Wille, a senior HRW researcher. HRW has said such plans would violate international law which forbids arbitrary detention without trial. Baghdad wants to avoid a repeat of Camp Bucca, a US-run detention center where ISIS leader Abu Bakr al-Baghdadi, who is still at large, expanded a network of radicals during the US occupation of Iraq. Prime Minister Adel Abdul Mahdi said earlier this year there was a "difference between families which were trapped by ISIS as hostages and families who accompanied the terrorists. "We will respect human rights but ... there need to be careful security measures so (ISIS supporters) cannot leak back into society." Existing displacement camps already host some 450,000 Iraqis, and aid groups say even there they are sometimes denied documentation and face the prospect of never returning home for real or alleged affiliation with ISIS. Whether in camps, guarded housing complexes or at liberty, the longer refugees who lived under ISIS are marginalized, the more likely they are to be exploited by militants seeking to regroup and expand an insurgency, Western officials say. Salih and his family left their native Salahuddin province as Iraqi forces drove the extremists out in 2015, moving to Mosul for two years, then Qaim, ISIS’ final Iraq stronghold. "When Qaim was attacked we paid smugglers to get us to Syria," he said. "We want to go home, but wont do that as long as the militias are in control there. Theyll kill us or lock us up. They destroyed my farmland - I saw a video of them setting it on fire," Salih said, an account Reuters was unable to verify. Iraqs Migration and Displacement Ministry, which deals with returns and camps, did not immediately respond to requests for comment on the plans for the displaced or alleged rights abuses. As tens of thousands of Iraqis and Syrians languish in camps, some say they would accept ISIS’ return if it meant they could go home. "I didnt leave my city because of ISIS," said Khalid Fassal, a 30-year-old Syrian man at al-Hol who fled the town of Albu Kamal when Syrian regime forces recaptured it. "But when the regime came, I didnt want to take chances. "If ISIS came back, wed stay," he said.