IBM and the challenge of systems design

Summary:In the past, systems could be designed for a single purpose. Although every system design is a set of compromises, it was possible to partially optimize a system design for a specific purpose. Today, system architects have been forced to try to build "do-everything-well" machines. IBM's announcements show its long history in system design.

Database and storage intensive tasks

Database and storage intensive tasks make different use of processing power processing intensive or highly interactive tasks. The system is going to be focused more on moving data into and out of the system and less on computational work. In this case, having a greater number of processors is a better idea than having a single very fast processor.

Database and storage intensive tasks do some processing and then send the result out to the storage system. The results are also going to be sent out to the end user's device, but getting things into and out of storage comes first. Reducing or preventing the need to touch the storage subsystem all of the time has lead to database-focused systems to rely on huge memory configurations. Data is kept in memory as much as possible to reduce the impact of storage latency. As NoSQL databases and Big Data applications become increasingly important, the design of the memory subsystem becomes even more important.

While the memory system plays a huge role in database and storage intensive applications, eventually data must be sent out to the storage subsystem. This increases the focus on intelligent storage systems that pre-fetch data and bring it into a system's memory.

When NoSQL and Big Data applications are considered, high speed network access means the difference between acceptable and unacceptable performance. The data is no longer shuttling between the processor, memory and storage. In this application, the data is moving from system to memory, to network, to memory to system and only occasionally to the storage system. When it does move to the storage system, very large amounts of data will need to move into and out of the storage system quickly.

Topics: Hardware


Daniel Kusnetzky, a reformed software engineer and product manager, founded Kusnetzky Group LLC in 2006. He is responsible for research, publications, and operations. Mr. Kusnetzky has been involved with information technology since the late 1970s. Mr. Kusnetzky has been responsible for research operations at the 451 Group; corporate and... Full Bio

Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.