There's also convergence in a more specific sense in that hardware is becoming commoditised — a generalisation of course but broadly true. For example, storage networks will eventually run on Ethernet, just like the rest of the datacentre.

Anything that makes management easier — less skilful, if you like — will be welcomed

And there's convergence because the plans of many vendors will result in bringing those three core disciplines under one roof from a management perspective.

For example, Windows Server 2012 majors on new automation and management capabilities. Network and storage vendors both talk of — and of course offer for sale — products that provide a single pane of glass that allows you to manage everything in the datacentre.

And we've figured out how to reduce downtime resulting from equipment failure. It's expected, so we route around it. Consequently, vendors talk of reducing management effort and human mistakes through automation. That's a powerful argument since outage as a result of hardware failure is now, generally speaking, less likely than finger trouble, as many high-profile outages attest.

The CIO certainly wants convergence since he or she wants to contain staffing costs, to comply with what the business demands: to do more with less. Whether it's a single pane of glass or some other process that reduces the need for human intervention, anything that makes management easier — less skilful, if you like — will be welcomed.

Extracting the best price for skills

Yet there's tension in the air. If you're a Microsoft Most Valued Professional, if you're a Cisco-certified engineer, if you're an EMC Proven Professional, what you want is to be able to maximise those skills. You want to be able to use them to the full, and to extract the best price for them — to enjoy them, in other words.

Even you might want a single pane of glass. It can make life easier, I'm told, but I'd be glad to hear of how that's true in the real world.

But does it mean the start of a deskilling process? If it's like most other jobs, then it probably does. Can the tide be held back? Should it?

Or is this all a storm in a teacup, and the likelihood of these increasingly complex technologies becoming operable by — shall we call them less well-trained folk? — is slim?

ARM — originally the Acorn RISC Machine — is a rare success story in the processor business whose time has arguably arrived, courtesy of the huge and ramping demand for low-powered CPUs to drive battery-hampered mobile devices.

The future of mobile chips is interesting, but it's their role in lowering energy usage that could have the biggest impact on most people

It's hoping its chips, whose design it licenses out, will become the basis of tomorrow's servers, driven by the increasing imperative to cut the costs of powering and cooling the datacentre.

It's not going to be all plain sailing, though. As an article in Wired about Intel and the decline of Dell, HP and IBM in servers points out, Google et al are showing little sign of switching from the x86 architecture. To do so would be tantamount to throwing away millions of years of developer effort to code for a new hardware platform.

Instead, they're taking a new route. Rather than buying servers from Dell, HP and IBM, whose combined market share of the server business has been dropping, they're building their own.

So far so unsurprising. We've known for years that Google takes this approach. What is interesting is that Intel has confirmed that eight server makers now vie for a share of the same pie — 75 percent of the server market in 2008 — that was previously sewn up tight by the big three.

Server builders and customised machines

More interesting still is that Google is not alone. Many server builders are making customised machines for the likes of Facebook, something Intel obviously knows since it's selling CPUs to Google — the only web company to buy direct, according to Wired — and to the smaller server builders.

They are making systems that include such features as a Facebook-designed friction hinge, making server maintenance quicker and easier.

And all include Intel processors, which Intel reckons can get close to the kinds of power usage that ARM offers — just 15W — but without the humongous hassle of changing platforms.

Market share figures in this area are a bit speculative, since Intel doesn't reveal its sales figures, and market research companies such as IDC — the main data source — cannot or find it harder to count private sales to Facebook et al.

The future of mobile processors is interesting, but it's their role in lowering energy usage that could have the biggest impact on most people, whether or not they know it.

For a discussion of how this issue might play out, including a discussion of AMD's potential role, take a look at Jack Clark's piece Intel draws battle lines.

Your first reaction may be, "Of course not." But many find it's justified to pay for the extra performance, typically just for the data that's worth speeding up. For example, in a datacentre, you might speed up just a database's working set or, on a personal machine, just the OS and applications. Having the OS or a complex application such as Photoshop pop up in seconds is worth the price of entry.

As Crump points out, most cost-justification in the enterprise for SSDs is done on a cost per IOPS, not cost per GB. That way, you compare it with the cost of short-stroked disks, of whose capacity you might only be using 50 percent. The rest of the disk space goes to waste, along with the cost of spinning and cooling the things. SSDs score here.

According to Crump, they also score when you front-end a spinning disk system with an SSD cache but the performance calculation is more complex. Flash-storage vendors like to talk about how many extra IOPS you get and the time you save when using SSDs as a cache but never, in my experience, point out that this technique only works when the proportion of cache hits reach a certain level.

If there's a cache miss, you're into negotiations between disk systems:

Have you got this bit of data?
No.
OK. Do you know where I can find it?
Try over there.

Not pretty or desirable. But throw in some deduping and compression and, with luck and a following wind, you might claw back some half-decent capacity savings, though nowhere near the volumes you save when backing up multiple VMs or client PCs. That might bring the justification of SSDs closer into line.

In the longer term, and as flash memory prices fall, I wonder if we might not see more systems using some form of execute-in-place technology, where the distinction between memory and storage becomes more blurred, in much the same way as it is in mobile phones. I appreciate that's more in the way that it's described and used, rather than in reality.

In the meantime, flash-memory vendors mostly sell systems that use a disk interface between the computer — that is, the CPU and memory subsystem — and storage. Not only does this practice add upfront cost, as Crump points out, but it also adds to your opex, as you have to house and power those disk controllers, as well as keep their drivers up to date.

Alternatives are emerging, such as that from the likes of Fusion-io, whose PCI-E-connected flash memory storage goes some way towards mitigating this problem.

But for most organisations, this approach is both way too expensive and potentially limiting — just a single vendor sells the product so you'll need a strong justification.

Instead, when considering SSDs to boost performance, ask if it really is cost-justified, and whether your datasets are suited to the kinds of usage that SSD vendors envisage when showering you with shiny PowerPoint presentations.

The problem comes when a company grows beyond the limits of a few NAS boxes and needs more storage. Lots more storage.

Today, NAS comes in a standalone box, and there's not a lot you can do with it other than store data. You can't scale out because the box doesn't permit it, and you can't scale up without buying a ton more storage to upgrade the disks, even assuming that is a simple operation. If you've got more than a basic configuration, chances are you won't want to go too far down this road.

Scale-out NAS boxes

So what's the answer? Storage vendors at the top end are providing scale-out NAS boxes — EMC Isilon is an example. To add more storage, you add a box and it becomes part of the system.

It's managed from the centre, and allows you to enlarge your storage pool using a single name space, without having to shift terabytes of data around to balance the load across multiple boxes.

Feature sets in scale-out NAS are becoming richer — you're getting more products with synchronous replication

As well as reducing the amount of time spent on management per gigabyte, scale-out means savings on power and cooling too, as you're not running as many separate units, each with its own built-in infrastructure.

That's not true where most companies live. Here, you buy storage in discrete units and then have the increasingly tedious task of managing a growing farm of storage devices. If you don't manage them in this way, you run the risk of paying for storage you're not using, and overloading some boxes to the point where performance suffers.

This problem is particularly common where applications are tightly tied into one or two of the devices, and upgrading then becomes a major headache, perhaps also associated with downtime as you move the bits around.

Richer scale-out feature sets

However, there's some light at the end of the tunnel. Scale-out NAS has traditionally been aimed at organisations with a need for large volumes. Examples would be media companies, and those using scientific data, so large data sets have been prioritised over enterprise style features such as deduping, snapshotting and replication.

According to analyst Arun Taneja, founder and consulting analyst at Taneja Group, feature sets in scale-out NAS are becoming richer — you're getting more products with synchronous replication, for example — and those enterprise-level features are arriving in scale-out NAS products aimed at smaller organisations. And analyst Terri McClure from Enterprise Strategy Group believes that most data storage will be on scale-out NAS in the next few years.

Some would suggest pushing all that data into the cloud. But as we all know, the network is still neither reliable nor fast enough for mission-critical applications, although some cloud storage service providers are getting around this problem by using cacheing gateways at the edge.

Hmm, more devices to manage and cool — and this type of technology is still young, so most organisations will probably wait for version 3.

In the meantime, the time for NAS to get smart is overdue.

Quantum supplied the figures for the cost of hardware rather than a service and, of course, Quantum has a large footprint in the tape library market.

Additionally, as some commenters point out, at 10PB over five years, the volumes of data on which this calculation is based are huge. Most companies — small to medium-sized enterprises — are more likely to want to archive off maybe 10 to 50TB: several orders of magnitude less. At this level, Glacier is likely to work out cheaper. Much cheaper.

I could ask Quantum to do the maths for me for this volume of data but I see that Quantum has just launched its own Glacier-like service in the form of Quantum Q-Cloud.

Q-Cloud's pricing is identical to that of Amazon Glacier when storing above 72TB but, instead of sending your archived data directly to the datacentre, you stage it using a Quantum DXi deduplicating disk-based device, which then deduplicates and synchronises to a Quantum datacentre in the background. This has the advantage of allowing local retrieves, as long as the data remains recent.

This is brave stuff. You do need some more software and hardware components — available from Quantum — but you can also get your data back more quickly. Amazon is quoting several hours, while Quantum reckons you can have your data back much faster. Looks like Quantum is hedging its bets.

This won't be the last cloud archiving initiatives we see. With Amazon's entry, the bar has been raised, and it remains only to wait and see who else tries to leap over it.

This technology, mainly in the form of SSDs, has brought huge benefits to datacentres as well to laptops and desktops, primarily by adding speed through increased I/O and lower latency.

To the end user these improvements bring clear performance benefits, whether that end user is at the other end of the network or at the same machine where the SSD is installed.

From the datacentre perspective, solid-state memory has enabled storage managers to add another tier to their existing two-, three- or four-tiered models. Such models have been created to contain the costs of buying high-performance storage for all data, reserving it instead for the data that benefits most, usually databases and other transactional data.

The rest, such as Windows shares, tends to live on cheaper, slower storage, where an added wait of a few milliseconds is barely noticeable, if at all.

Flash memory has also enabled applications that could hardly have existed before. While flash memory vendors are eager to tell us about implementations of desktop virtualisation, the reality is that few installations of any size are fully up and running.

They say flash memory will help to change that, as its performance profile enables the high I/O demands of virtual desktops now to be met without breaking corporate piggy banks.

That's all very well, but you can only fully reap the benefits of new technologies if they're properly integrated with existing systems, and if they're directed towards business processes rather than just being faster.

Flash vendors are moving in this direction, not just because potential customers are asking for it but also to help differentiate themselves.

For example, one vendor of in-server flash memory storage, Fusion-io, specialises in flash memory installed inside the server, on the PCI Express bus, where it acts as a super-fast storage drive. It has recently announced software that allows server-side flash storage — and therefore datasets — to be shared across servers.

According to company spokesman Gary Orenstein, it speeds large jobs such as media transcoding and database transactions because its 6GBps bandwidth can return data to the server within 50-100 microseconds.

In relation to connectivity work over Fibre Channel or InfiniBand, Orenstein said: "Normally the SAN isn't the bottleneck, so most customers can just attach this system to their FC network."

Others suppliers, such as WhipTail with its Accela flash appliance, are focusing on developing technology that helps accelerate the specific read-write patterns of individual applications, which is a hard problem for applications that do more writing than reading, such as databases and VDI.

So while flash memory makes everything go faster from an end user's point of view, from the datacentre storage manager's point of view, it's a technology that needs careful integration with existing systems. If not, it could just end up being an unjustifiable expense.

Woz's main point is that: "I've come to a depressed state of feeling that I own nothing on the cloud and have no ability to keep things working the way they do." Elaborating, he says that cloud services operators can and do change and drop features without so much as a by-your-leave.

Because his cloud-based calendar could be messed about with, he issued his siren call for regulation of cloud providers.

"I believe that regulation applies to banks and that money lost due to no fault of your own is replaced, at least for large amounts. Why not for the cloud, as well?"

He has a point. Cloud providers are perhaps less likely to chop and change their service offerings to enterprises, where much more money is at stake and where continuity is probably the highest currency, but cloud provision is always going to be a movable feast.

Technology changes fast and services will inevitably reflect changes in the infrastructure, as cloud providers find it becomes less cost effective to deliver a service using older processes and technologies. This situation will change and, in this, I'm a bit more bullish than Woz, who says that he "expect[s] this sort of occurrence to get worse over time".

Tend to monopoly

That's because markets tend to monopoly. The companies at the top get fatter and will retain bigger reserves to handle technology shifts, delivering new services enabled by new technology while keeping older services going in the cause of continuity.

Larger enterprises also have more options. They can choose to host all or some of those services themselves in private clouds, retaining complete control over the infrastructure and the pace of change. Enterprises are also more knowledgeable, and more likely to do due diligence before opening their wallets.

All that said, though, I think there's a place for regulation in the cloud, especially for personal use. People are encouraged to use cloud as an alternative to storing stuff in their homes or wherever. The demand is in part a consequence of the burgeoning number of devices we use and carry about. You want that data available everywhere, irrespective of hardware platform.

Safety net

Given that much of that data is likely to be highly sensitive, and that cloud use has moved outside geekery and is now being pushed at the less technologically knowledgeable, a form of safety net seems increasingly appropriate.

It's analogous to the early days of telecommunications. Whether it's data protection in the form of encryption, whether it's a commitment to ensuring that certain safeguards are applied to personal cloud-based data, whether it's a guarantee that people can retrieve their data at any time and in a format that allows them to use it and move it to another provider, cloud providers should be held to account.

The banks touted market forces and self-regulation as the best protection for customers. Not so. The same applies to cloud providers.

If you then asked a vendor for a set of solutions to your problems, you might expect them to be partisan in their approach. If you get lucky, you could expect them to be more nuanced.

Sadly, when this precise exercise was tried at the recent NetEvents symposium in Miami, many of the vendors, when asked for solutions to a theoretical set of CIO-level problems, were depressingly on-message from their companies' rather than the potential customer's point of view.

Four network vendors gave a two-minute pitch on how they would fix the problem.

If there was a common theme, it was that the CIO needed to do server virtualisation first — as long as he or she wasn't planning to push the problem into the cloud, where, so it would seem, you have just recreated the problem but on someone else's infrastructure and with added remoteness. Not smart.

The conference session ended with two analysts, Nav Chander from IDC and Peter ffoulkes of TheInfoPro, summing up the vendors' offerings. Their verdict was that most vendors' answers were not clear, and that not all the problems had been addressed.

To be fair, given the limitations of a 50-minute conference session, the pretend CIO could provide few real-world details but vendors need to better at getting to grips with the underlying problem rather than presenting a generic solution if they are to win a CIO's trust.

Google's search engine went down in 2009 but, given the volume of traffic it handles, this is not a bad record over the last decade. So the company's technology is pretty robust.

The second question being asked by enterprise buyers of cloud services is: how does it work?

This is where we hit the sticking point. As highlighted in a previous blog, buyers of cloud services increasingly want to know what's going on under the bonnet. They want transparency as much as they want a good deal. Does Google give them that?

Google may be able to paper over the cracks in the answers it gives more than many, if not most, other cloud providers. This is because it can point to its record and its continuing position as the world's leading internet search engine, with 84 percent market share.

But potential customers will still want to know where their data is and what's happening to it. This Google cannot do, because it has never revealed the secrets of its technology and shows no sign of so doing.

There is an alternative: OpenStack, which Rackspace recently described as the Linux of the cloud world. It lets anyone create a cloud service, is backed by an open-source code base, and is already doing better in terms of contributions than Linux was at the same point in its history, according to Rackspace's chief executive Lanham Napier.

The obvious stay-aways from the OpenStack effort are Amazon and Google — proponents of proprietary technology both of them. When open-source software (OSS) first arrived, many pooh-poohed the idea. How could it make money? How could it be robust? How could it be trusted?

The OSS community has proven itself capable of developing technology that shoulders some the biggest loads out there, so there's no question that the model works. Now time for open-source cloud to step forward?

And how do you deal with the multiplicity and diversity of hardware and software that's left over from previous mergers and those sudden growth spurts, as in: "We must have 10 servers now, don't care what they are"?

One company, Arkeia, reckons it has some answers and has posted five tips for backing up virtualised environments. Most are self-evident — but perhaps only if you've been around the block a few times.

The backup specialist puts ease of use at the top of its list of considerations. This is sensible: the quicker and easier backing up is to do, the more likely it is that backups will happen, especially in smaller companies where resources are stretched.

Find a cross-platform backup product is the next tip, which means not just one that can back up different OSes, with all the various ways of locking open files and so on, but also one that can handle virtual and physical servers. After all, it's restoring the data to a usable state you care about, not backing up, so why make a distinction?

Arkeia also suggests using image-based backups. Now that block-based rather than file-based deduplication systems are becoming more common, this point too makes sense as you can dedupe the data that images contain just as easily as individual files.

However, you might like to consider an additional, file-based backup routine for directories containing critical files so that there are copies you can access quickly and easily without having to restore a whole OS, VM or snapshot.

Which brings us on to deduping, a no-brainer. If you're not to be swamped by the grind and expense of backing up 50, 500 or 5,000 copies of Windows, you need deduping technology.

The final point is to practise restoring the data. It's too easy to focus on backup and worry about restoring — when? When the proverbial encounters the air-moving device, that's when. Too late... Just do it.

The industry likes to bang on about big metal storage systems, facilities and features because that's where the money is. But companies come in all shapes and sizes, so I'll bet there's a lot who could do with a few minutes considering this advice on backups. Thoughts on a floppy disk, please — preferably one that's been backed up...

According to Corbelli, the UK's mid-sized companies — the bulk of his customers have between 100 and 1,000 employees — are getting stuck in what he calls bad relationships with their cloud services provider because they lack either the management and negotiation skills to handle problems with the relationship, or enough money to cover the cost of getting out.

The reason for their predicament is often because customers fail to do enough due diligence before awarding a contract, Corbelli said.

Soft issues

He discussed the amount of work that potential cloud customers ask Star to do before awarding a contract. In contrast to three years ago, when potential cloud users were interested mainly in facts and figures, speeds and feeds, today the main discussion points centre on 'soft issues'.

About half the negotiations revolve around issues such as people, governance, processes, and compliance. "It used to be all about just the technology," Corbelli said. "Now it's about who you are, your culture, your reason for being, the peer-to-peer relationship. People want to know if you can be trusted, if you're on the same wavelength. They want to feel that their CFO can call your CFO and talk to them as equals."

The reason for the extra caution and pickiness isn't hard to find. "These are hard issues. The technology is fraught with issues," he said. "People have tried to manage inhouse for years but have decided to look at an outside service. That's all shrink-wrapped around the SLA and other agreements."

"It used to be all about just the technology, Now it's about who you are, your culture, your reason for being." — Martino Corbelli, Star

As with most business decisions, the people side of things is a major factor. "You can tick the boxes but still not get the deal," Corbelli said. "It could be because of all sorts of reasons: financial, such as a venture funding with outside interests that means the company has to go down a particular route, or the CFO has relationships with another company. If we lose the deal, price is used as a deflector — you don't find out the truth after the event."

Mid-sized companies are increasingly buying enterprise-level services, according to Corbelli. "What resonates with the mid-market is accessibility to private cloud services," he said. "This is developing into a hybrid cloud approach."

Email

And the most common problem that organisations ask Star to resolve is — surprise, surprise — email. Internal email systems are often a tangled web, having evolved from an organisation's beginnings but added to through acquisitions and mergers, and branch offices doing their own thing. The cost of sorting that lot out can be more than the cost of asking an external provider to start again and do it centrally.

The moral of the story seems to be to choose your external service provider carefully, check them out fully, and be sure you feel comfortable with them before signing on the dotted. And be sure you know where the exit is, and how much it will cost you to turn that handle.

In March, I wrote about Xsigo, and how it is bidding to centralise the arrays of cabling that infest server racks. Instead, you install an Infiniband card in the server and connect it to a box at the top of the rack. Inside Xsigo's Fabric Director box is where the I/O cards - storage and networking - now reside and, to the server, they appear local.

One of the benefits of this approach, which is effectively a virtualisation of the server I/O stream, is that it allows you to switch I/O cards, as well as attach and detach them in software, all of which fits well with orchestrated virtual infrastructures. Other benefits claimed include improved performance and reduced operational expenditure, because there are fewer cables to manage and it's quicker to alter configurations. In theory, it could also allow you to get away with deploying fewer expensive components such as HBAs.

But Xsigo doesn't have the field to itself any more. NextIO has joined the fray with its vNET I/O Maestro. The company also promises to boost I/O and reduce costs -- but then, why else would you buy the concept let alone additional boxes that you'll need to manage, something neither vendor spends much time talking about?

NextIO's sales and marketing VP Mike Heumann reckoned that his company wasn't in competition with Xsigo but Cisco and Brocade -- it's good to be associated with the big beasts in the jungle. However, he tacitly admitted during our conversation that Xsigo is his main rival. NextIO's advantage over its longer-established rival is, according to Heumann, that potential customers liked Xsigo's idea but not its entry price point. So NextIO is undercutting Xsigo by focusing on deployments of one rack at a time, not four or five racks at a time. "They're at a higher level," he said.

This is how it works. "We run a PCI-X link between the server and our box, which then accepts all the adapters that are seen by the server as local," Heumann said. "You can then decide which adapters are presented to the servers and what percentage of bandwidth they can use, which you can change dynamically. It means you can do anything, including teaming and fail-over, that you would do locally. It also means you don't need to change your governance policies."

The typical cost per rack is about $75,000 for 10 servers, plus the cost of the PCI-X cards, which Heumann reckoned cost about $100 each.

Interestingly, Cisco is aware of the presence of Xsigo and its potential threat to its datacentre hegemony, and has reportedly alerted its sales and marketing teams to weaknesses in the virtualised I/O approach in general and Xsigo in particular.

Both companies have funding from the usual range of Silicon Valley investment houses, which suggests that there is a need for the technology. Let battle commence...

On the face of it, and if you're taken in by the hype, cloud is the next wave of technology before which we should all genuflect. But lets get this into perspective. Research forecasts predict that cloud computing will only represent about 10 percent of global enterprise technology spending in the longer term, with most of it going on the kinds of stuff it does now.

The one-to-one relationship which is the simple way of presenting cloud will be rare, by which I mean the idea of one company having one relationship with one cloud provider. It'll be a lot messier than that, as cloud providers will offer different services at different price points and in different geographies. Even smaller companies, who are the most likely to adopt one provider for all their IT services, are likely to buy in services from elsewhere as well. Dropbox anyone?

Expect to see cloud providers specialising in particular areas in terms of services, verticals, and geographies. Yes, geography still matters, as you do still care about where your data lives. Heard the story about how, under the US Patriot Act, the FBI ripped out an entire rack of servers in a datacentre just in order to grab one of them? All the service provider's other customers in that rack were seriously affected even though they had nothing to do with it. And there are legal stipulations on data location too.

Underneath all this there's a lot of centralising going on. Amazon Web Services (AWS) along with Rackspace is one of a handful of big cloud infrastructure providers. There's there's likely to be fewer of those over time, and they'll be big. On top of their infrastructure, other companies are building what the trade jargon calls over the top (OTT) services. For example, Dropbox's data lives on AWS kit. Who knew?

On top of the likes of AWS, service providers will increasingly build mash-ups of services from other providers. This is happening already of course. You can already find applications that bring together data from diverse sources and present it in new and useful ways. It will get easier as the tools mature, making this development methodology faster, cheaper and far more prevalent. All this needs open APIs to allow developers to grab data and make use of it, and organisations such as The Open Mashup Alliance are working in this direction.

But cloud, while an important element of any company's IT thinking, needs to be kept in perspective. Use it when it's appropriate, not when it's not, and don't be swept along by the hype. In other words, research it.

Of course, there are still some companies, like Apple, who really don't get the power of cloud. Ever tried syncing an iPhone to two separate computers? Can't be done without a good deal of chicanery, goat sacrifices and widdershins dancing, as the experience of a good friend attests. If instead, Apple used a cloud-based repository as Google does for Android...

As well as offering shares for Windows and Mac (CIFS and AFP) today's NAS systems can now be found offering thin provisioning, deduplication, replication and snapshotting. If you include systems such as FreeNAS, which is based on Sun's ZFS, it even offers all that for free -- although you will of course need a hardware platform if you don't already happen to have hanging around a VMware ESXi server with a couple of gigs of memory to spare.

But looking at finished products, Infortrend's EonNAS range offers deduplication, a rare feature in SMB-level products, while Synology and Drobo allow you to use the extra capacity of disks larger than the smallest in a RAID array.

One thing to watch out for though, when considering whether deduping is worthwhile in a production environment, as opposed to backup where it clearly makes sense, is the potential performance impact. The way in-line deduping works is by grabbing a chunk of incoming data, hashing to produce a value and comparing that to previous chunks. If there's a match, don't store it, store a pointer instead.

This is fine when backing up as, in the normal run of things, you don't actually care how long a backup takes as long as it's done by the time people come into work the next day. So an extra few minutes are neither here or there.

When working on production data however, you do care, as time taken to dedupe is perceived as sluggishness and latency. I know of one example where just this happened, prompting the vendor in question to rewrite its deduping engine.

And since hash tables are usually held in RAM, the solution in many cases is to add more: a lot more. But a better one is write a smarter hash table algorithm, so that the most likely hits don't need to be searched for on disk but out of RAM. Or even SSD, if there's one fitted. There's more about this topic here if you're interested.

I've discussed open source storage before, suggesting that the barriers to use of OSS for storage were more technological than cultural, given its adoption rate. Evidence that the adoption of OSS for storage is happening and that this is coming true is not hard to find. Among the latest straws in the winds is Nexenta's release of a virtual desktop software system.

Desktop virtualisation is increasingly popular, if researchers are anything to go by. Reasons aren't hard to find. The ending of Microsoft support for Windows XP mandates that many corps upgrade. And since migrating from the old OS to the new isn't a straightforward 'stick it in the CD drive and run the script' jobbie, it makes sense to think about alternatives to just buying a few pallet-loads of new hardware at a cost to the corp of squillions.

There's a lot more to say for it: centralisation of management which means better security as a result of more up-to-date applications and OSes, reduced costs of hardware, and better control over costs.

According to Nexenta CEO Evan Powell, a big barrier to adoption of virtual desktops is complexity, which drives up the cost of installation and testing. Another barrier is the high level of storage performance required, especially for write operations, numbers and sizes of which are particularly high for desktop OSes.

Powell, incidentally, remains passionate about open source, and sees Nexenta's return of code to the community-developed OpenSolaris as core to the business. But his company has also just launched NexentaVSA for View, a system for deploying virtual desktops into VMware environments.

According to the company, it converts the deployment process that normally takes up to 150 steps into a four-step process. It is said to automate the the processes of configuring and rebalancing the desktop infrastructure, provide better visibility into performance issues, and allows you to use high performance storage such as SSDs as caches to speed up write operations.

Of course, this system really only supports one type of virtual desktop infrastructure - there are plenty of other ways of providing desktops remotely other than stuffing them into VMware-hosted VMs. How popular they are though isn't, in my view, that important: what is important is that the method delivers the desired outcome.

But if the VMware route is the chosen one, Nexenta's new system could help - especially if you combine it with Virsto's eponymous VMware-based software that effectively sequentialises (and so speed-boosts) the highly random I/O load that virtual machines -- and especially desktops -- desktops present. The V company has just launched another SKU of its product, so that it now runs on Microsoft Hyper-V, VMware ESXi and now Citrix Xen.

Further thoughts as they occur but feedback meanwhile welcomed.

The latest contributor to this never-ending debate is Steve Watts, co-founder of the tokenless two-factor authentication vendor SecurEnvoy. According to the company's latest press release: "Commenting on reports that a security developer has concluded that password-creation policies are the enemy of secure passwords, SecurEnvoy says that the fundamental issue is that conventional ID/password security is now coming to the end of the line as far as security is concerned."

Ignoring the somewhat tortured press release-speak English, we have here a vendor taking issue with this piece on ZDNet about security developer Cameron Morris. Morris has created an open-source tool called Passfault that predicts the time it takes to crack a specific password -- there's more about it in the article.

Watts' beef is that password strength is a moveable feast and not very well understood, so the answer is obviously something else: two-factor authentication, I suspect. The problem here is that he's confining the scope of the issue to highly controlled enterprise environments. Most two-factor authentication vendors do that too.

There's no doubt that two-factor is stronger than a password alone, especially one that needs to be strong. Weak passwords are clearly to be avoided but their convenience is hard to beat, as strong passwords are hard for most people to remember.

But two-factor authentication may no longer be the solution to this conundrum. Internet services and personal devices like smartphones now find major uses within the enterprise. The reality today is that the division between enterprise and personal environments has all but evaporated. In the course of their jobs, people increasingly access their personal services at work using their personal devices. And enterprises cannot mandate two-factor authentication for access to Facebook, for example, which might well be the chosen method of communication of a key supplier, or a way of communicating with potential customers. All FB wants is a password, and it's not alone.

So I think it's time for security experts to accept that the password is here to stay. As we all know, convenience trumps security any day -- just look at what happens in small companies all the time: sharing of passwords, open passwords, no passwords at all in some cases -- are you screaming on the ceiling yet?

Instead we need to find ways of making passwords work. Personally, I've been through a number of solutions, including a tiny portable password generator that will never, alas, be developed as a cross-platform utility. This has driven me towards KeePass, an open-source password safe which relies on a password to open it -- and you can even boost protection with two-factor authentication. It's cross-platform and has a good ecosystem of plug-ins and other support. Use it together with a cloud service and your passwords are available anywhere.

There are others like it out there but I moot that something like this may well be the way forward in many circumstances.

A survey commissioned by Microsoft (whose involvement was not revealed to the respondees) found that small to medium-sized businesses might gain a fair amount from adopting cloud computing. The survey, conducted with organisations in Hong Kong, India, Malaysia, Singapore and the USA, reckoned that most organisations viewed a move to the cloud as positive - but that they saw the cloud provider as responsible for the security of their information.

The company said: "32 percent say they spend less time worrying about the threat of cyberattacks. U.S. SMBs using the cloud also spend 32 percent less time each week managing security than companies not using the cloud. They are also five times more likely to have reduced what they spend on managing security as a percentage of overall IT budget."

A Microsoft blogger on the company's Trustworthy Computing site said that results from more surveys held in different geographies was due soon.

There's a number of questions here, not least of which is what the respondents assumed was the meaning of security. Did they mean backups and disaster recovery, which many small organisations might rightly view as security, or do we mean what the security industry means, ie freedom from attack by unauthorised individuals and software? I suspect the latter but there may be room for doubt.

However, this also raises the question of who is responsible for their data. Contrary to respondees' assumptions, a cloud provider will only secure your data if you pay for that service, and it's unlikely that an SMB would do so unknowingly.

That said, I'm somewhat sceptical these days, given the numbers of SMBs using cloud services -- they've been much faster than larger organisations in doing so -- whether data security is as big an issue for SMBs as many assume. Experience teaches us that convenience and cost trump security every time, so offloading to the cloud is more likely to be seen as a cost benefit to cash-strapped SMBs than a security risk.

I'd further hazard a guess that many small businesses are more likely to assume, incorrectly, that their data is not important enough to warrant an attack, a view stemming from a misunderstanding of the nature of the threats out there.

It'll be interesting to see the results of surveys from other areas but, in times of economic turmoil, I suspect they won't be very different.

It's a bit like having an alternative supplier for commuter trains in the rush hour although -- if the analogy will stretch this far -- having to go to an alternative station to catch one.

This of course lies at the heart of the promise of cloud computing: enabling the agile enterprise, bringing flexibility and so on. But what does it actually mean? And is it really as simple as it sounds?

This train of thought was triggered by a lunchtime conversation with Marten Mickos, CEO of Eucalyptus, which builds cloud software.

The first thing that springs to mind is that applications don't work without data, and moving data around is expensive in terms of time and money, especially if you're paying a cloud provider for storage and bandwidth.

The time element is critical. Imagine: your demand spikes, so you need to shift that excess computing load into a cloud provider's facilities. But the database behind the application is huge and certainly won't be moved in the time available.

Instead, you move that database into the cloud ahead of time. Now you have the problem that you're paying for that storage even when you don't need it, and you need to find a robust mechanism to keep that database in sync with the live data in your private cloud.

What's more, that data is now potentially at risk if it's sensitive, such as credit card details. Is the cloud provider certified under PCI-DSS guidelines to hold such data? And can you recreate the applications' environment in the public cloud such that it can attach to that data in a seamless manner? If not, are the tools available to make it so?

But hold on! I thought that cloud was supposed to reduce costs not start running up a bill for capacity I don't actually need right now.

You would hope that cloud platforms such as Eucalyptus could take some or all of the pain out of this scenario. If so, all well and good. But before you start swallowing the hype about the cloud's instant flexibility, it would be worth taking a closer look at a real world scenario.

I'd be interested to hear real world examples of cloud bursting.

Greenpeace has just released a report entitled How Clean is Your Cloud? In it, the environmental organisation highlights how the decisions all of us make as user/consumers are increasingly having an impact on technology's energy usage and CO2 emissions.

It also looks at and takes aim at some of the big names on the Internet, and the way that their technological innovation is not being matched by similar advances in energy efficiency.

The report notes that the growth of cloud s computing is driving growth in the numbers of datacentres but that: "Three of the largest IT companies building their business around the cloud – Amazon, Apple and Microsoft – are all rapidly expanding without adequate regard to source of electricity, and rely heavily on dirty energy to power their clouds."

On the other hand, Yahoo and Google garner praise for continuing "to lead the sector in prioritizing access to renewable energy in their cloud expansion, and both have become more active in supporting policies to drive greater renewable energy investment."

And perhaps the least surprising finding is the way that some companies continue to claim green credentials "despite a continued lack of transparency and very poor metrics for measuring performance or actual environmental impact."

So the big problem with technology companies is, as you might expect, a lack of visibility into energy usage, especially data that would reveal the numbers and size of their facilities. Vendors cite commercial confidentiality as a reason for hiding their energy sources.

Does it all matter in the great scheme of things? A 2010 report from Greenpeace calculated that: "The combined electricity demand of the internet/cloud (data centers and telecommunications network) globally in 2007 was approximately 623bn kWh. If the cloud were a country, it would have the fifth largest electricity demand in the world."

This is somewhat misleading as the Internet is clearly not the same thing as the cloud. In fact, cloud spending is, according to some figures I've seen, set to amount to between 2-10 percent of all enterprise IT expenditure. It's going to be more than that for consumers but the environmental organisation needs to be careful to avoid double-counting.

That said, Greenpeace proposes that that datacentre managers should report their CUE (carbon usage effectiveness), which was adopted by the Green Grid over a year ago, in addition to PUE. This means the datacentre would report not just the ratio of energy usage within the datacentre between computers and environmental equipment, but carbon usage too, so taking into account energy source efficiency.

Greenpeace's report also includes an interesting geographical analysis of both where major datacentres are located and their energy sources.

There's lots more in it: take a look...

We don't yet know who's going to win this shoot-out but we do know that the daddy of systems vendors, IBM, has finally joined the circle, which so far has consisted mainly of Cisco and HP, with Oracle standing by as a bit of an understudy.

Starting from the middle of May, IBM will start selling PureSystems, which it describes as an integrated system for the datacentre with: "the flexibility of general purpose system, the elasticity of cloud and the simplicity of an appliance tuned to the workload-fundamentally changing the experience and economics of IT."

It consists of IBM's own Xeon- or Power-based servers, Storwize storage, and 10GbE networking, plus storage connectivity networking from QLogic and Brocade, and application and infrastructure software from the usual suspects, including VMware and Red Hat -- and of course IBM.

Pure's server element is a new set of 10U chassis and horizontally oriented blades, a system that, according to this report, is more flexible than IBM's existing BladeCenter system. The Storwize kit provides locally accessed tier storage and other virtualisation features.

There's a lot there, and much of the detail of these systems, whose entry-level price is said to be around $100k, has yet to be revealed. There's a little more detail here from IBM.

It's clear though that the approach IBM has taken with Pure sits between those of Cisco and HP, using mostly IBM products, some existing, some new, along with some third party hardware and software. In contrast, HP has produced everything itself, arguing that the 'one throat to choke' approach is best, while Cisco has assembled what it calls best-of-breed components, mostly from third parties.

What's also interesting is what it means for datacentre managers. Cisco and HP have had the converged datacentre block market to themselves for a while but this means little: this is not a market with a fast turnover. IBM will have been planning and developing this system for at least 18 months, probably more.

What's more, it has previous. There's no company that understands more about developing and selling systems than IBM, and it's been at it since the beginning of time -- in computer terms, you understand.

That's why you can be sure that, if it's a success, IBM will make shedloads out of software, service and support. It's even reportedly including an IBM engineer in the price. It's not clear if said individual will live in one of the boxes, popping out as and when.

In a way, IBM is betting the company on PureSystems. While IBM is diversified enough that it won't bring the company down should enterprises not buy enough PureSystems to defray the no-doubt considerable development costs, for IBM to make a failure out of selling them would be embarrassing. At the very least.

So you can expect a lot of sales effort and now IBM has had time time to identify what it might see as the opposition's weak points, I would expect Big Blue to have got it right.