Only a rip-and-replace
David and I agree that data center security is a problem. Where we differ is in the approach to remedy that problem. I believe that any change you can make to a data center should be done incrementally, except for security: only a rip-and-replace overhaul of data center security. Data centers have historically had excellent physical security, but have fallen tragically behind in network security.
Over-the-network attacks, such as DDOS attacks, are but one area of vulnerability for today's data centers. To mitigate network attacks, data centers need to replace old hardware with new, smart devices and better monitoring and alerting. With individuals and businesses moving to cloud computing and cloud storage, data centers need to move quickly.
Attackers don't attack incrementally, nor do they plan their attacks over several month's time. They attack in bursts and en masse. Only an overhaul of network security and constant vigilance can combat these attacks. A methodical approach to security will only make the problem worse not better. An overhaul is expensive and labor-intensive, but you have to weigh those costs against the cost of a single data breach. The costs to customers, to a company's brand, and to the data center itself is too great to use any other approach to the problem.
Unfortunately data center customers are far too vulnerable and are far too important to incrementally protect them from existing and upcoming threats. I've heard the analogy that to eat an elephant, you have to do it one bite at a time. But that analogy doesn't work with outdated security, because attackers have already seized the elephant and have gathered a herd behind it. Only a complete data center security overhaul can stop the stampede from breaking down the door.
An ongoing process
One thing that Ken and I agree on, is that the root cause of many security issues is Soylent Green; that is, people. And getting people to change their behavior is almost always an incremental process. People don’t like change and the more significant the change is, the more resistant they tend to be. But this has been an issue for IT as long as there has been an IT department to complain to, and not one limited to data center issues.
While I firmly believe that careful incremental changes are the safe way to update your data center security model to provide minimal disruption to your primary task of getting business done, there will always be situations, such as a massive breach of your security or the discovery of fundamental flaws in your security protocols that require wholesale changes. But this should be the exception, rather than the rule.
Security is an ongoing process, which should constantly be under evaluation with proactive changes and adaptations being made to keep your data center ahead of those who wish you harm. Needing to do a sudden, major overhaul to your security means that, in most cases, you have failed to provide the level of security that you should have already been providing, be it IT security or physical security.