Optimizing data center security: Overhaul or incremental changes?

Moderated by Larry Dignan | June 23, 2014 -- 07:00 GMT (00:00 PDT)

Summary: Our experts take a hard look at the state of data center security.

Ken Hess

Ken Hess

Overhaul

or

Bit by bit

David Chernicoff

David Chernicoff

Best Argument: Overhaul

38%
62%

Audience Favored: Bit by bit (62%)

The Rebuttal

  • Great Debate Moderator

    Are my debaters ready?

    We'll be starting promptly at 11am ET (8am PT). Welcome, readers: Starting at 11am ET, this page will refresh automatically each time a new question or answer is posted.

    Posted by Larry Dignan

    Ready to rumble


    Ken Hess

    I am for Overhaul

    Standing by...


    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    OK, first question:

    What's the state of data center security today?

    Posted by Larry Dignan

    Existing data centers are playing catchup...

    ...but they're losing ground quickly. There are two primary areas of data center security: physical and network.

    Physical security is very good to excellent. Typically, there are multiple layers of security that one has to traverse before entering a data center: An external security gate, a secure circle lock (man trap) or badge-in access door, badge-in access to the server floor, retina scanners, and maybe even fingerprint scanners--all related to physical access to the data center facility and to the secure server areas.

    There are loopholes to physical security that are too lengthy to discuss here, but some of the obvious ones are parcel deliveries and janitorial services.

    Network security is a larger problem for data centers, especially in multi-tenant ones. Multi-tenant data centers present the largest problem because there are no enforced security standards from client to client. Single tenant data centers face security problems as well but at least a single client data center can enforce a security strategy across all systems.

    Ken Hess

    I am for Overhaul

    At the moment, data center security is in flux

    There are so many new technologies and techniques coming at the data center operator that security is pushed down the list of concerns. This, itsel, is a fundamental problem. Security may well be the number one issue that has to be dealt with because of its across the board impact, but dealing with so many new products, technologies, and services is keeping data center IT scrambling just to stay current.

     

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    What do you see as the biggest threats facing the data center?


    Posted by Larry Dignan

    Ill-prepared for DDoS attacks

    Other than viruses, worms, and various types of malware, the biggest threats are DDoS attacks, insider threats, and mobile devices, especially Android-based mobile devices.

    I think that data center security is generally ill-prepared for DDoS attacks. And every data center is susceptible to insider threats, regardless of physical security measures. But the greatest threat is the new onslaught of mobile devices allowed by BYOD scenarios and by the absolute number and diversity of those devices.

    Ken Hess

    I am for Overhaul

    The top of the list is software-defined-everything

     s shown with the recent Joyent crash, it is really easy for a software issue to take down an entire data center. As using SDN et al becomes more common, the potential for a security flaw exposing the data center to malicious attack will grow exponentially.

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    Where does physical security fit in the data center planning?


    Posted by Larry Dignan

    There are a few breakdowns

    As I've stated before, physical security is very good to excellent in most data centers. Data center designers, planners, and security personnel take very good care of the physical security of their data centers. However, as I've also stated, there are a few breakdowns in this area.

    Data center planners need to separate delivery areas from the main facilities and to disallow access by anyone other than data center personnel to main facility loading docks. Third party staff members such as janitors, maintenance, and support staff need to have standard background checks and limited access to the various areas of the data center. There should never be any "All Area Access" to facilities, unless those accesses are used under strict supervision by data center staff.

    It's more expensive to provide great physical security for a data center, but you have to compare that cost to the cost of a single breach, which can run into the millions of dollars. It's basically an ounce of prevention vs. a pound of cure mentality.

    Ken Hess

    I am for Overhaul

    Physical security concerns are dependent on the type of data center

    For a normal facility, the standard physical security process in place that limit access to the building, control availability to external resources such as generators are usually sufficient. With certain datacenter trends, however, physical security becomes a major issue. The top two are lights-out facilities and micro-datacenters.

    With a lights-out facility, access control and actual physical security measure that prevent smash and grab attacks, such as those that have happened to telco datacenters in the past, are serious significant concerns.

    With micro-datacenters, the nature of the design makes them easy targets for physical theft and physical access that can enable the security compromises the console access can allow , which means that planning to prevent these very different threats needs to be a major concern in the deployment of these units.

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    Weakest links?

    Where the weakest links in the data center security chain?

    Posted by Larry Dignan

    Any security chain has the same weak links...

    ...people who ignore, bypass, or defy security. The weakest links are people. Security is a pain. We see it as a necessary evil and, after a period of time, the weaknesses are that the people who enforce policies and those who have to comply with them will relax or bypass the more painful measures. It's human nature and bad guys know this too. And knowing this, I'm surprised that there aren't more major physical security breaches at data center locations.

    After working in several data centers over the years, it's easy to find the loopholes and exploit them. But, it's only those who comply with the security measures that have to deal with the pain of them. Security chains are setup by people who understand security and compliance. The measures are setup to prevent problems, but they don't address sometimes often obvious security holes and the people who allow them to exist.

    Ken Hess

    I am for Overhaul

    Anywhere end users have access

    This might sound a bit facetious but the reality is that as data center portals are deployed to enable users to do more on their own with less direct assistance from IT, the security problems are more likely to be introduced from a client computer than anywhere else.

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    Software defined data center?

    How does the move to the software defined data center change security planning?

    Posted by Larry Dignan

    A software defined model will alleviate some of the human-caused security problems...

    ...but certainly not all of them, because underneath all of the software is still all the hardware--hardware that has to be installed, wired, physically maintained, and decommissioned.

    A software defined data center will cut down on some of the traffic to the data center because provisioning new servers, network, and storage will be performed over the network. It will increase the need for better network security, so there will only be a slight shift in security planning, not a major one.

    Ken Hess

    I am for Overhaul

    Makes everything more complex

    The SDDC makes everything more complex from the security perspective. The potential of unexpected security problems due to minor changes in the datacenter definition will require significant effort to be expended in testing prior to rollout and monitoring after deployment.

    As different aspects of the data center move to a software defined model (e.g., networking and storage) the potential for security issues increases. As FPGA technologies become more prevalent, the security model will need to be one of the first issues addressed with any change to the datacenter infrastructure.

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    Does the move to converged infrastructure and systems...

    ...improve security or make for an easier target?

    Posted by Larry Dignan

    There is a third option:

    It won't have any effect. I don't think that converged infrastructure will have any real effect on security, physical or over the network. Converged infrastructure is more of a marketing thing than it is an actual solution for anything. It's an attempt at a new twist on a pre-packaged, modular solution, but you still have all the same components included, so there's no real difference in security.

    Ken Hess

    I am for Overhaul

    While the glib answer is 'yes'...

    ...the truth is that there will be both advantages and disadvantages.

    On the plus side there are greater vendor resources dedicated to making sure that the entire infrastructure they provide is secure. On the down side, a higher profile means a greater likelihood that more malicious attacks will be targeted at that infrastructure as it would apply across multiple locations rather that the custom, home-built design of the traditional data center.

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    Does converged infrastructure make your respective cases harder or easier and why?


    Posted by Larry Dignan

    No difference

    Converged infrastructure makes no difference in my case because, as I wrote in the previous question, we're not talking about something new here, it's something old that's repackaged and sold as an integrated solution.

    Ken Hess

    I am for Overhaul

    I think it makes the incremental approach moot

    If you have decided to go with a converged infrastructure offering form a vendor, you’ve gone with the overhaul method by default. Future security modification may be incremental, but that is now out of your hands and the choice of the vendor of your infrastructure.

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    Series of upgrades?

    At what point and time frame would a series of upgrades of data center components be considered a security overhaul vs. a slower move?

    Posted by Larry Dignan

    Upgrade vs policy change

    I'm not so sure that it would be an upgrade as it would be a change in technology and a change in policy. An overhaul would be replacing old technology with newer, smarter technology, such as intelligent intrusion detection devices, smart honeypots, self-healing network hardware (routers/firewalls/load balancers), and better software.

    An incremental change would leave vulnerable areas more vulnerable as you upgrade or replace components. An overhaul (mass replacement) over a period of a few months would fit my definition. Incremental is more of a we'll do this in the first quarter, this other thing in the second quarter, and it extends the project to a period of 18 to 24 months. These days, that sort of slow approach isn't going to work.

    People who attack data centers, websites, and company assets don't typically hire a project manager and methodically plan an attack over a period of months. And the response to those attacks or in their prevention can't be slow either.

    Ken Hess

    I am for Overhaul

    I think that his point is specific to each data center

    While replacing the security infrastructure wholesale is clearly an overhaul, upgrading that infrastructure as you add new services still fits the incremental model.

    If your data center upgrades are things like switching completely to software defined networking, for example, and updating security of other components to match the new completely redone networking infrastructure, you’ve moved beyond incremental to overhaul.

    There is a point that the data center IT operations management team will realize that their issues are beyond band aids and patches and that a full overhaul of the security infrastructure may be necessary.

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    Should we just end this debate...

    ... and tell everyone to go cloud and forget about it? 

    Posted by Larry Dignan

    Ha, that's pretty funny...

    ...but also not a bad idea for some companies. A lot of people still fear the cloud, however it's a good option. What I think we forget when we talk about these technologies is that under that cloud is infrastructure in a data center. Someone has to maintain it and I've pointed out that multi-tenant data centers have their own problems.

    Unfortunately, there's no single right answer for every business. Sometimes there's no single right answer for every business unit within a single company. One business unit might be able to migrate 100 percent to the cloud, while another might never feel comfortable doing so.

    Ken Hess

    I am for Overhaul

    Absolutely not

    The cloud is "a" solution, not "the" solution.

     

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    I was being a bit cheeky with that last question...

    ...but how does the cloud change the security equation? And isn't there a good argument that your cloud provider will have better security than you will?

    Posted by Larry Dignan

    The same problems

    Well, you certainly wouldn't have to perform any overhauls if you used a cloud provider. The provider would be responsible for security--at least for server infrastructure, network, storage, and physical. You'd still be responsible for application, possibly operating system, and services. Some of their network security would help, but not prevent yours apps and services from being vulnerable.

    We would hope that providers would have better security, but it's not necessarily so. They have the same problems that everyone else does with all the security aspects we've discussed here.

    Ken Hess

    I am for Overhaul

    The selection of cloud service providers goes far beyond security...

    ...though they are certainly a big target for hackers and others of ill intent. So at least part of the issue becomes the tradeoff between being a bigger target and that target’s greater investment in assuring that their security is strong. And there is the fact that you will never know exactly how secure their model is. What the vendor doesn’t share can’t be used against them so there is a limit to how much information they can realistically make public about their security infrastructure. So while they may have a better security infrastructure than your own internal operation, they also may be no better at protecting their resources than you are.

    You are also giving up control over the end-to-end data path your information travels, which means that you can only make presumptions of the level of security once that data moves outside of your closed environment. You are also potentially exposing for more information as you transport it from within your data center to the secure confines of your cloud provider.

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    Gentlemen, we have time for one more question:

    Where does security play in the decision to maintain hybrid data center infrastructure?

    Posted by Larry Dignan

    It plays a very important role...

    In fact, it might be the weakest link in the non-human security chain. The problem is that if your data center is too diverse in its infrastructure, you have a support problem that translates into a security problem.

    For example, if you're trying to maintain multiple vendor server hardware (IBM, HP, Dell, Sun), then you have to also maintain a constant vigilance for each type of system and its security needs. For system administrators, this is a nightmare, but it's also very typical of data centers. Hybrid infrastructure is sort of the norm. It's also highly dysfunctional and the source of a lot of problems--security and otherwise.

    With hybrid infrastructure also comes a horrifying array of operating systems on all that hardware.

    To specifically answer your question: Security should play the ultimate role in every decision we make in the data center. I know that's sweeping and broad but we must have some sort of consistency to maintain a high level of security.

    This is another area of overhaul. To maintain a high level of security, you should (as much as possible) standardize on a hardware vendor for your server infrastructure. Patch often, maintain a high vigilance level, and keep your systems updated. Being behind on patch levels is what system hackers look for.

    And don't assume that new hardware is less vulnerable than your seven year-old clunkers. Patch the new stuff before it goes into production.

    Ken Hess

    I am for Overhaul

    Security requirements don’t go down

    Even in a hybrid data center infrastructure your security requirements don’t go down. You are still responsible for maintaining the security of the data and the infrastructure for the portion of the infrastructure under your direct control. While you may determine that you can trust your provider’s security, you are likely to be monitoring your cloud resources via logging tools as well as any monitoring that the provider offers specific to their service. You need to make sure that security remains a high priority for that monitoring, and that you don’t make unsupported presumptions about the security of your data.

    David Chernicoff

    I am for Bit by bit

  • Great Debate Moderator

    Thanks to Ken and David for a Great Debate

    And thanks to our readers for following the action. Look for the debaters' closing arguments, delivered tomorrow, and for my final verdict -- issued Thursday at 2pm ET.

    Posted by Larry Dignan

Talkback

14 comments
Log in or register to join the discussion
  • Too vague / situational . . .

    "Optimizing data center security: Overhaul or incremental changes?"

    The question as stated seems too vague and general to be answered.

    Whether you need to overhaul things or just make incremental changes tends to be situational. Was the system built with security in mind to begin with? Is the system modular enough for the desired changes? Is there a lot of legacy code that may need to be rewritten? How large of a rewrite would we be talking about, and how expensive would it be?

    To me, the very nature of this question tends to be highly situational. I don't think it's something that can really be answered in a generalized context.

    I guess we'll see where the conversation goes.
    CobraA1
    Reply 275 Votes I'm Undecided
    • "The cloud" is data centers

      "Should we just end this debate...

      ... and tell everyone to go cloud and forget about it?"

      "The cloud" is data centers too, lest we forget. For all I know, when you say "data center," you could very well be talking about Amazon's or Microsoft's or Google's or any number of "cloud" providers. They have to be concerned about this stuff as much as anybody else who runs a data center.

      No, "the cloud" is not a magical collection of fairy dust. It too runs on actual machines at actual data centers, and is subject to the same questions.
      CobraA1
      Reply 250 Votes I'm Undecided
      • Correct

        Pushing your data to the cloud just means you're delegating management to an outside firm, which probably doesn't care nearly as much about the security of your data as you do.
        John L. Ries
        Reply 260 Votes I'm Undecided
      • the cloud ???? LOL

        the biggest (at least one of them) misconceptions out there !!!!! you are right..."cloud" = "data center" !!!!!! no difference
        neal tech
        Reply Vote I'm Undecided
  • Have a Cup of Coffee?

    While David is having his cup of coffee (just to think about it), his datacenter is probably being torn apart. We had monitored the network interfaces and hackers are attempting to gain access at the rate of 100s of attempted attacks per hour. True, you have to be careful but, if your system is not secure, you might as well shut down your applications. That way, at least the fines won't cost you tens of millions of dollars (depending on governance of your data).
    hforman9
    Reply 243 Votes I'm for Overhaul
  • Strictly situational.

    I'm pretty sure Target is going through an overhaul.

    How much of that overhaul gets watered down by the time it gets to the grunts is the question. From their point of view, it might actually be a bit-by-bit change, whereas from the management point of view things are being radically changed.

    For most things, I believe an overhaul is needed.

    The problem starts with acquisition... If security is not considered a primary function, there will only be a patchwork security available. Not reliable, not secure, and just as vulnerable as most sites are now.
    jessepollard
    Reply 250 Votes I'm for Overhaul
  • Depending on your situation.

    If your data center security is good to great than step-by-step is good. But if your data center is stuck in dial up era then you will need a overhaul of the security. Each one of these methods has it advantages & drawbacks and you will need to determined either one or an mix of the two helps your security situation.
    As for a overhaul of the security it make long longer time to implement since it will require compatibility and testing for a period of time to get work properly before it get implemented and that is where bit-by-bit parts comes in, you implement most crucial parts first and then implement other parts later. For the budget conscious, the bit-by-bit method is best way as to to have large budget outlay as in the overhaul method. However, overhaul method is good if have the budget & could implement all of the systems all at once.
    Again, it depends on your situation.
    phatkat
    Reply 266 Votes I'm Undecided
  • In the end...

    ...the system has to serve the needs of users who need to get their work done; thus if you tear everything out and start all over again, you force people to learn a whole new way of computing which can be highly damaging to productivity in the short term and may even prompt a rash of "do it yourself" (the inevitable result of a loss of confidence in the computing staff), which will make security worse, not better.

    The other problem is that overhauls tend to be put off until an opportune moment, which might never come; incremental things can be done quickly and they tend to add up.

    There is occasionally a good reason for a complete overhaul of the system, but not very often.
    John L. Ries
    Reply 245 Votes I'm for Bit by bit
  • the issue goes into the roots of o/s design

    one should go back to the Tannenbaum/Torvalds debate to understand the roots of the issue. Read Bruse Schneier: "Complexity os the Enemy of Security". study history: what were these systems designed to do? read the news: 2014 is on track to be the Biggest Year yet for Hackers. if you are already running an o/s with better security then turn to your CMS and DB software: does this stuff only run programs you have set up and checked out or will it run anything a hacker throws at it ? remember: a hacker is going to put the CMS or DB on a de-bugger and step through it, examining every crack in the fence... ...

    a band-aid ain't gonna help.
    Mike~Acker
    Reply 264 Votes I'm Undecided
    • Nobody was suggesting band-aids

      And making sure the security is right on the programs being run suggests the incremental approach rather than the overhaul. There may be times when a radical reworking is necessary, but most of the time, all that is required is to make sure proper protocols exist and are followed; and that security measures be properly tested (to include penetration testing) and fixed when they fail.

      An overhaul probably isn't necessary. A proactive staff working to properly secure the data and educating their users on safe computing practices definitely is.
      John L. Ries
      Reply 226 Votes I'm Undecided