Foreign nationals suspected of association with terrorists often cannot be formally charges. The U.S. seems to be using deportation as a method of disposing of these cases. Read the LA Times story.
In the days before 9/11, sending a seemingly nice person back to, say Pakistan, seemed like harsh punishment for nebulous rules violations. Now, it’s the best option available. As the Times points out, there’s too much expense and wasted time involved in tailing these people. They cannot be held in custody, either, realisitically, for any length of time without running into other problems.
There is no constitutional protection for these people’s immigration status. Better safe that sorry, as the cliche goes.