FBI throws a scare into datacenter service providers

FBI throws a scare into datacenter service providers

Summary: How should law enforcement separate the wheat from the chaff?

SHARE:

In a story reported yesterday evening by the NY Times at http://bits.blogs.nytimes.com/2011/06/21/f-b-i-seizes-web-servers-knocking-sites-offline/, the FBI decided to take down activity from a suspicious IP address by seizing three enclosures full of servers from a hosting Facility in Reston , VA, used by DigitalOne, the hosting company, based in Switzerland, that was being used by the target of the FBI investigation.

The only problem was that the three enclosures worth of servers apparently included the sites for many more customers than just the one being investigated and DigitalOne is responding to the outages reported by those customers by letting them know that the FBI has those servers and there is no way that they can check on them, or do anything else with them, for that matter.

Despite their interest in just a single DigitalOne client, the FBI's actions have affected "tens" of clients, according to DigitalOne CEO Segej Ostroumow. The FBI has not yet commented on their actions nor have they provided any way for the customers they were not interested in to recover their server data.

So what does this mean to cloud and internet service providers, and more importantly, their customers?  What happens when the FBI decides that a fellow customer of a cloud service that your business uses needs to be investigated and shut down? Does the distributed nature of the cloud mean that the FBI will shut off and confiscate every server and storage device potentially involved in their investigation?

If you are a current colo customer, it would appear that there is, at the moment, a potential problem for you if you don't have full racks, enclosures, or your own suites. The actions of a completely unrelated customer of your datacenter host can have consequences that put you out of business, if their servers simply happen to co-reside with yours.

I'm given wonder if any of the servers in the confiscated enclosures were mirroring or backing up to a different physical site, and if they had been, would the FBI have raided that second facility to confiscate that equipment, on the off-chance that their targets data was being passed along.

This governmental action brings to light a new take on privacy and security, especially with the cloud.  Will the government continue to make what appears to be"guilt by association" assumption related solely by proximity, that they have the right to damage and destroy unrelated businesses while performing their investigations?  Or will someone step forward, admit that this was as bad as serving a no-knock warrant at the wrong address with devastating results and require that the FBI, and any other government entity that could potentially do this, examine their policies and come up with ways to prevent this kind of collateral damage.

Topics: Servers, Data Centers, Government, Government US

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

22 comments
Log in or register to join the discussion
  • Hysterical! Our government at work...

    doing what it does best; screwing up.

    Regards,
    Jon
    JonathonDoe
    • RE: FBI throws a scare into datacenter service providers

      @JonathonDoe
      I'm not sure you can say they're "screwing up" here.. It's a valid investigation, and just as the FBI may close your whole street if there's an active incident in a neighboring house, more then just the intended target of the investigation might be impacted/ inconvenienced by that investigation.

      Could they have asked the CoLo to identify only those machines owned by that one customer? Sure, but who's to say the night-shift guy at the CoLo facility isn't involved with the criminal activity?

      Who's to say that the bad guy's computers aren't using neighboring computers to store some information on them? Afterall, it's possible/likely that the router/firewall ACLs could be more relaxed for computers in the same subnet/cabinet, and the bad guys could have used that advantage to offload some incriminating evidence onto someone else's servers.

      You wont find me defending government too often, but there's PLENTY of more agregious cases of law enforcement overstepping their bounds on the Internet than this one, and I think any company that puts their servers "in the cloud" thinking that they can just forget about them and not worry about what NEW risks may exist in this new computing paradigm is just putting their head in the sand.

      If those were MY servers, I would definitely have multiple servers in multiple datacenters in case one whole set of servers were to go offline for ANY reason. Fire/Flood/Earthquake/whatever would be higher on my list of reasons for doing this than the possibility of the FBI taking my servers, but the resulting care I would take would protect me from this anyway. Hopefully those "tens of clients" did similarly.
      172pilot@...
      • RE: FBI throws a scare into datacenter service providers

        @172pilot@... <br><i>Could they have asked the CoLo to identify only those machines owned by that one customer? Sure, but who's to say the night-shift guy at the CoLo facility isn't involved with the criminal activity?</i><br>Come on, they're not going to be asking the night shift guy at the Colo about their investigation, they're going much higher than that. If the night shift guy is complicit that will come out. The point is that the FBI could have taken just the target company's data without bringing down 10s of other clients who aren't a part of the investigation.

        <i>Who's to say that the bad guy's computers aren't using neighboring computers to store some information on them?</i>
        I don't think you understand how data centers/Colos work. All companies are segregated. The Colo facility provides power and space. It may sometimes provide your WAN connection. Basically all companies in a given colo are segregated from each other. I've never been able to get into any other colo clients' equipment and I'm confident that no other clients of our colo can access our equipment. In the data center world, just because your servers are in the same physical building as my servers means nothing. They may as well be on different continents as far as data integrity are concerned.
        swmace
      • Although I agree with your points ....

        @172pilot@...
        .... this still looks like the blowing up of an entire apartment building because of one or two tennants.
        kd5auq
      • RE: FBI throws a scare into datacenter service providers

        @172pilot@...
        Exactly right. If a "black hat" has access to one company's servers, it's entirely reasonable they may have access to others. Until you know how they gained access you have no idea if they used the same method to compromise other servers/storage/etc.
        DevRandom
      • RE: FBI throws a scare into datacenter service providers

        @172pilot@... Jesus, what an ass you are.
        endlr@...
  • RE: FBI throws a scare into datacenter service providers

    Clearly there's more than meets the eye here. Digital One is high on the "interest" list of the FBI as well as the "target" of the investigation. Digital One is well aware of the surreptitious activities on its servers and is obviously part of the larger investigation. It is well known that most hackers of these groups are techies at service providers such as Digital One, as well as many bogus "clients" found on the servers of Digital One, such as those companies that were also "affected" in this incident and where NO ONE has ever heard of before, until now - very suspicious. I also don't need to remind you that physical evidence can be found on the actual hard drives/modules in these servers that can be retrieved only with the servers in hand. Deleted material still has a residual presence in the physical drives that can only be recovered through physical analysis. Also, only a stupid hacker would use exclusively his/her own resources such as their own domain, etc, especially when given the fact that their "job" is to readily hack multiple servers/IP addresses in pursuit of their goals. Time will reveal that more than a few "clients" of Digital One had their servers compromised by the target group in pursuit of their goals. It's so easy to pin fault on the government for taking actions against criminals we may have sympathy for, just like when the FBI killed Dillinger and Bonnie & Clyde - heroes of many during the depression. When they [FBI] don't do all we expect in an investigation, it's their fault again. They can never win. Time will tell. In the end, only the bad guys hate it when the FBI does it's job. Who are the bad guys? Those targeted in the investigation and those criticizing the FBI for doing it's job. Bad guys suck and if you're one of them, I hope they get you too!
    kenalexruss
  • RE: FBI throws a scare into datacenter service providers

    The FBI isn't the only using the nuclear option. Several years ago, the IP address of my smtp server was in a cluster of IP addresses that had been identified as sending spam. I regularly had trouble with people getting email from my domain because IP ranges were being blacklisted, not individual addresses. Basically I was hanging out in a bad neighborhood. The problem wasn't resolved until I changed hosting companies.
    R.L. Parson
  • Give your bleeding heart a break

    When you factor millions of stolen user accounts and passwords (sega, sony, etc) against collateral damage to 5 - 10 unrelated server customers, no one in their right mind will complain.

    You don't ban crosswalks just because someone's child got hit in one. You accept that 1) you need crosswalks 2) that some small proportion of kids will get hit in one.

    It's no different. If you locate your equipment in a facility used by criminals, there will be some proportion of collateral damage. And you also need to expect that the mixing of facilities in a cloud environment will allow that to happen.
    croberts
    • RE: FBI throws a scare into datacenter service providers

      @croberts While I agree that collateral damage is inevitable in a cloud environment, I believe that was exactly the final point of the author. Why would I send all (or any part) of my data to a cloud environment that is subject to potential risk. My job as IT Mgr is to prevent downtime, and protect the company data. I make every reasonable effort to prevent this. So if the sanctity of the cloud is compromised and security of my data is now out of my control, I would be negligent to utilize a cloud service whereby security is far less I can provide in-house.
      Heritage02Rider
    • RE: FBI throws a scare into datacenter service providers

      @croberts while I agree with you statement, I do believe this is exactly the point the author was trying to make. Why would I place all (or any part) of my data into a cloud based service? As the IT Mgr, my job is to protect company data and secure it as best as I can. If the sanctity of the cloud is now compromised by indiscriminate authoritative actions and security is out of my control, I would be negligent to utilize a cloud service for my companies data. Thereby making the cloud a less than reasonable solution for most companies who rely on access to their data.
      Heritage02Rider
  • It is time to treat the virtual world like the real one

    Law enforcement and law makers treat the virtual world as something special. It would be much better if they treated it like the real world.

    In the case above if these were paper files in file cabinets, the FBI would need a strong case against the ISP to take more than the one company they were interested in.

    There is more than this, I know hackers employed by local law enforcement and the FBI who routinely try to hack into systems for their bosses without warrants. If this was breaking and entering real world files the bosses would need to find a new line of work.

    The rights surrounding search and seziure by law enforcement has been well developed for the "real world" it is time to add cyberspace to that world.
    oldsysprog
  • RE: FBI throws a scare into datacenter service providers

    Time will tell what is the real FBI's jobs
    Bonek suroboyo
  • RE: FBI throws a scare into datacenter service providers

    "<b><i>...have affected ?tens? of clients...</b></i>" WOW!!! What an outage! What 20, 30, 60?! Maybe that explains why I never heard of DigtialOne!
    The Rifleman
    • RE: FBI throws a scare into datacenter service providers

      @The Rifleman
      Sure, it probably doesn't matter to you if your company wasn't brought down by the FBI for no reason. What if those 10 customers were Google, Apple, Microsoft, HP, Amazon, etc. What if one of those 10 customers was your business that is now down with no explanation and no recourse? How do you guys not see a major problem with this?
      swmace
  • RE: FBI throws a scare into datacenter service providers

    I think some of you are missing the bigger picture.
    Apple, Google, Microsoft, etc. are all planning to offer wide reaching "Cloud" services including data storage.

    What if one of Google's customers was a criminal?
    Would you be happy if the FBI confiscated all of Google's servers?
    Of course they probably couldn't carry them away physically, but they could certainly sever all the connections.

    It normally takes the Government years to sort out it's court cases.
    Could you live without your business/personal data for years?
    Would you still have to pay Google's monthly fees for all of that time?
    lehnerus2000
    • RE: FBI throws a scare into datacenter service providers

      @lehnerus2000 agreed. The cloud definitely looks less appetizing now than it did before this event. I was considering some cloud services, but now am hesitant to use any that may actually store my company data.
      Heritage02Rider
  • RE: FBI throws a scare into datacenter service providers

    Anyone who thinks this was overkill doesn't understand the problem and likely hasn't dug into the NY Times story about the raid and the targets - Lulz Security Group and Anonymous. (After you read the initial story linked above, also see: http://bits.blogs.nytimes.com/2011/06/20/hackers-declare-war-on-government-agencies/ ).

    These are nothing less than online terrorists if not anarchists. If they were operating through this datacenter then the FBI showed amazing, perhaps *excessive* restraint in not seizing the entire facility. Nothing there should be considered safe until proven to be uncompromised and even then I'd move my business out of there after this.

    The response and tone from the DigitalOne CEO suggests to me that rather than the FBI not understanding what they were doing as he suggests, either he doesn't understand what *his* datacenter was doing or he is complicit in their activities.
    DevRandom
  • RE: FBI throws a scare into datacenter service providers

    What it comes down to is this.
    In order to prosecute, the FBI needs the actual machines
    that the site is running from; otherwise the defense can claim manipulation or the like.

    To do what they need to correctly, they need to do the following, assuming that datacenter is not a co-conspirator.

    1) Seize the machine. Have the machine put in maintenance mode.
    2) Make a non-volatile copy of the entire contents of that machine.
    3) Allow the datacenter to use that copy to "move" the other clients onto new hardware.
    4) Take original offline and transfer the IP address to the
    new machine which has all the clients except the one that is being taken offline.

    This is the method that would allow the integrity of the evidence to be intact and minimize the damage done to 3rd parties.
    richard233
    • Yes, but...

      <p style="text-align: justify; margin-bottom: 1em;"><a href="http://www.zdnet.com/tb/1-98847-1920091">@richard233</a> The real problems with <em>that</em> idea are:</p><ul><li style="text-align: justify;">it would require competence, care and diligent effort on the part of the LEOs; attributes which historically have been rarely demonstrated with regard to technology;</li><li style="text-align: justify;">given the above, the low-level folks executing the raid wouldn't be able to answer the Big Cheese's phone call asking "Is it done?" 45 seconds after the raid com&shy;menced; and <em>especially</em></li><li>because doing so would demonstrate concern by the Government for the rights and <em>actual</em> security of innocent bystanders, which is expressly against post-Constitu&shy;tion&shy;al Government policy.</li></ul>
      Jeff Dickey