P2P storage: Can it beat the odds and take on the cloud?

P2P storage: Can it beat the odds and take on the cloud?

Summary: In a world where IT is centralising, some companies have been trying to involve users by producing clouds based on peer-to-peer technology architectures. The past has shown it's not an easy trick to pull off.

SHARE:
23

All around us the Googles, Facebooks, Microsofts and Amazons are building datacentres so they can store companies' information, allowing organisations to spend less on their on-premise IT equipment.

According to the digital cognoscenti, this breed of cloud computing is the way of the future. But what if there was another way to create a global cloud, one that didn't involve cementing the dominance of global technology companies like Microsoft, Google and their ilk? Would you use it?

Symform thinks you would. The three-year-old company told ZDNet this week that, as of 30 November, it is storing petabytes of information across 175 million files in a global peer-to-peer cloud. This data is not being stored in datacentres - as it is in the Amazon, Google and Microsoft clouds - but on the drives of servers, desktops and NAS boxes in 160 countries across the world.

The company is far from being the first to try to put the spare compute and storage capacity of people's machines to work, and judging from those who have gone before it, there's no easy path ahead.

P2P's mixed bag

Peer-to-peer architectures for computing have had variable success over the last decade.

There have been successes, like the distributed computing programs of SETI or Folding@home which use the spare compute cycles on people's machines to search for aliens or sequence proteins, but there have also been failures.

One would be Wuala, a P2P file storage technology developed by researchers at the University of Zurich that was spun off into its own company and subsequently brought by consumer storage giant Lacie. However, shortly after being bought, the peer-to-peer technology was shut down and the system was re-designed to operate from Wuala's datacentres.

Another service based on similar technology was All My Data, but that ran into funding difficulties and shut in 2010.

The majority of Symform's users are IT professionals or IT service provider companies

"The problems in all of these cases is an average consumer is not sophisticated enough and doesn't have enough data," Praerit Garg, the president and co-founder of Symform, says. "The complexity of the problem they tried to solve was too high."

To that end, Symform makes users sign the equivalent of a service-level agreement that sees them agree to provide 80-percent uptime from their Symform hardware over a two-month period. The majority of Symform's users are IT professionals or IT service provider companies. This strategy works because IT professionals are more likely to want a cloud data storage alternative to Amazon Web Services or another major cloud than a consumer, Garg says. 

These companies and individuals are "afraid of cloud because it marginalises their role. Here is a system they can be part of", he says. 

But will P2P be the success that its proponents hope?

"The challenge with P2P storage products is that the cost of storage is falling precipitously, making P2P storage often no more cost effective than cloud storage solutions," the chief executive of a cloud storage company who wished to be anonymous told ZDNet. "I have nothing against the model, it just hasn't worked out well in the past."

Topics: Cloud, Networking, Storage

Jack Clark

About Jack Clark

Currently a reporter for ZDNet UK, I previously worked as a technology researcher and reporter for a London-based news agency.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

23 comments
Log in or register to join the discussion
  • Where do I start?

    "All around us the Googles, Facebooks, Microsofts and Amazons are building datacentres so they can store companies' information, allowing organisations to spend less on their on-premise IT equipment."
    The corporate objective is NOT to allow customers to spend less ... but to lock in increasing revenues for the corporation and make customers spend MORE.
    If you look at the architecture of e.g. Google you will see it consists of carefully designed network commodity computing. The idea is to minimise Google's costs ... so that technology savings can be enjoyed by the company, with some small concessions to customers who can't do any better because ALL corporations charge excessive amounts for individual items.

    "According to the digital cognoscenti, this breed of cloud computing is the way of the future."
    I call that crowd 'corporate sheep'. They are everywhere on ZDNET. They might as well be employed by the marketing arm of the corporates, for all the free thinking exhibited in their writing.

    "But what if there was another way to create a global cloud, one that didn't involve cementing the dominance of global technology companies like Microsoft, Google and their ilk? Would you use it?"
    That's is a rhetorical question, right?
    Who wants to pay over the odds and be locked in?

    "Symform thinks you would [not want to be locked in and overcharged]."
    They were right. I signed up immediately after the first article on ZDNET from Robin Harris ... and bought one of the first HP Microservers available in the UK to run the symform software. Instead of MSFT's Windows Home Server crap.

    "Peer-to-peer architectures for computing have had variable success over the last decade."
    Rubbish!
    P2P has been so successful all major IT and media corporations would be delighted if it were banned completely. Indeed they have tried so to do with ACTA et al.The technology is so powerful and successful it has the capability to destroy their businesses in short order.
    What we have not seen is a legal way to marry P2P for the mutual benefit of a company with modest aspirations serving honest customers seeking value. symform look like they might achieve that goal. It is telling that symform was formed from ex-MSFT employees: MSFT could have done this years ago but chose to release skydrive with its 7GB limit instead.

    "The problems in all of these cases is an average consumer is not sophisticated enough and doesn't have enough data,"
    The sophistication part is true ... but it could be designed away if a corporation so chose.
    It seems to me that if e.g. MSFT, instead of breaking with OEM's and offering crap like Windows Home Server, continued on the trend of Storage Spaces/File History ... and built in warning lights for disks that needed replacing ... and all systems came with TWO disks (say) ... we would arrive at a balanced position for all parties. When everybody fights their own corner, it is left to the hapless consumer to come up with a secure design.

    "... doesn't have enough data ..."
    Baloney! A network with lots of 'small' nodes is ideal (cf Google search design). I keep reading about the beauty of the scalability and cost advantages of clouds ... but when I say 'so my tiny single PC in gazillions of tiny PC's will be no problem then', I get a sharp intake of breath and 'well these economies only apply if you have 300+ PC's'.

    P2P works increasingly well with more and more small chunks of data!!
    "This strategy works because IT professionals are more likely to want a cloud data storage alternative to Amazon Web Services or another major cloud than a consumer, Garg says."
    What this means is that those who understand IT know they are being ripped off by the global corporations and want something better. It also means that ZDNET in general fails to enlighten the average consumer and explain adequately the inherent dangers of adopting public clouds :-(

    "The challenge with P2P storage products is that the cost of storage is falling precipitously, making P2P storage often no more cost effective than cloud storage solutions,"
    Pure unadulterated nonsense.
    Global corporations often use very expensive 'business class' components ... because that's what you do if you are in business ... without regard to the resilience built in by an ARCHITECTURE using cheap commodity components. They completely ignore the principles of RAID outlined in 1988!! The I stands for inexpensive by the way.
    Please provide a cost comparison of symform storage v AMAZON storage to justify your argument.

    I predict that there will be at most a handful of replies to this thread, which is central to the future IT landscape ... and hundreds of comments on the sagacity of MSFT removing the Start menu for Windows 8, or APPL screwing up their map app. Nor will this critical concern ever merit a ZDNET debate ... when instead we will be 'treated' to the corporate bleatings of Bott v SJVN about mundane issues like UEFI boot and Mary Jo's rejoicing that the Surface RT can run Notepad :-(
    jacksonjohn
    • Passionate points well argued

      johnfenjackson@
      thanks for your thorough reply to my piece. Thought I should make a couple of things clear
      i) for the start, I was trying to imply less spent on on-premise IT = more spend in a locked-in format with cloud companies, sorry that wasn't clear.
      ii) Yes the following was a rhetorical question.
      As for the rest of your points, yes I agree.
      My position on cloud computing is that it allows for a form of highly centralised control by a select few companies ( http://www.zdnet.com/facebook-google-welcome-to-the-new-feudalism-3040093418/ ) and my cloud utility series.
      I agree that this is something that merits debate, why do you think so few people care to discuss it?
      Thanks very much for commenting!
      JC
      As for the cost comparison - working on it ; )
      Jack Clark
      • Same ol'

        "I agree that this is something that merits debate, why do you think so few people care to discuss it?"

        1. The general level of understanding in posts and replies is too shallow .... from ZDNET staff and readers.

        2. Most of ZDNET bloggers appear to have , or act as if they have, a limited remit. So Ed Bott and Mary Jo Foley are generally supportive of MSFT ... even when the company screws up, or its corporate policy is manifestly anti-customer.

        3. ZDNET bloggers appear to be rewarded by clicks rather than quality, so we see much flamebait on controversial issues, rather than well thought out posts on key but difficult topics.

        4. ZDNET encourages a polarisation of opinion along party lines, much like political parties, so issues that are grey are treated as binary ... and respondents line up as if pushed by three line whips in a parliament. cf the recently introduced Great Debates. The result mirrors the ineffective American political system, where one side proposes a measure and the other rejects it. More likely is the situation where there is a clear answer but all ZDNET debates are hung parliaments. I have no idea how a newcomer could make sense of the conflicting views.

        5. ZDNET is too industry friendly - I prefer the attitude of The Register in the UK, who rightly treat major corporations as if they are operating just the wrong side of the law and purely in their own corporate interest: that's far more accurate (though of course not always true on all subjects).
        jacksonjohn
      • Feudalism captures the current position nicely ...

        ... unfortunately most ZDNET bloggers think they are 'free' in the American dream.
        We are increasingly in the grip of global corporations.
        Most ZDNET bloggers appear ignorant of the fact.

        Dream on. (Not).
        jacksonjohn
  • sorry but i dont get it

    im not getting it, why would i want to spread my data across a p2p network? why would i want to let someone else store their data on my system?

    when the same data is wanted by many people such as in a song or movie, ya i get that?
    when the data needs the same exact process run against it i get that
    but for just storage of data that is only usefull to one how is it a benifit to anyone to use p2p?
    if i have to buy storage why not just buy for myself?
    buying more storage than you need has never been a smart thing to do, prices fall so fast?

    off site backup seems to be the only thing... but why wouldnt i just buy extra drives and keep the backups in my safe deposit box, storage unit or any of the other places

    help me to understand, please?

    thx
    bob
    conobs
    • Couple of reasons

      1) gives you dropbox-like functionality where you can access your data from multiple devices
      2) instead of paying for access to data storage you can donate space from spare capacity on your existing equipment
      3) redundancy - didn't mention in piece but data is encrypted and split into 96 fragments which are distributed across global network but you only need 64 fragments to perfectly reconstruct file, so the data resilience/reliability is quite high
      any other questions just let me know
      best
      jc
      Jack Clark
      • sorry, im still not getting it

        ok, so who will this help? who has a fat enough pipe (to make this practical) that dosnt have better already?
        why is it better than me hot swaping an SSD or regular HD and mailing it to a friend?

        i can spend 20 dollars and have it overnighted back to me?
        if its a large amount of data it will take longer to move the data across the internet than to overnight it
        if its under 128 GB why wouldnt i wear it arround my neck?

        p2p dosnt make sense here
        conobs
        • does automation mean anything to you?

          you are welcome to continue "hot swaping" a drive and mailing it to your friend. how often do you do that though? weekly? monthly?

          i'm sure your users won't mind all that data loss when you need to do a disaster recovery and your friend mails you a hard drive from last month haha

          having an automated solution that is constantly backing up changes immediately to the p2p network seems better to me. I'd like the latest versions of my files to be restored when it comes to recovery...

          also, check out symform's hot copy feature (http://www.symform.com/our-solutions/key-features/hot-copy/), it is a constantly updated snapshot of your data at whatever location you want. your "friend" could have that set up so all changes are always synced to his place. then he can mail you the drive in a disaster, but with the most recent changes, not last months :)
          jbw976
        • I think I've got it - you tell me?

          If you want to protect your data, you need at least two copies.
          Maybe you have skill, buy two disks and try RAID 1.
          That's awesomely better ... and covers you unless:
          - your computer 'explodes'
          - part of your house burns down
          - you are robbed
          - when something goes wrong you are on holiday
          I'm an old codger and those incidents have all happened to me, in some cases more than once! I went to Italy on holiday two months ago. There was a sizeable power cut at my home while I was away.

          So you decide you need an off-site system (cloud?)... and the missus ... and the children ... are all clamouring for two disks now that they understand. You've also read that as disks have grown larger the recovery time and probability of a successful restore starts to become a problem if you use RAID 5. And all your family, friends and neighbours ... whose PC's you are maintaining ... wouldn't know RAID from their arse ... and if you asked them to leave a PC on all the time would recind your membership of the green party. They are always out when you call for your emergency backup disks ... after you've driven a distance. You know that RAID 6 would solve some of the problems ... but not all ... and a RAID 6 controller costs
          jacksonjohn
        • part II to solve ZDNET error

          ... and a RAID 6 controller costs
          jacksonjohn
        • part II to work around ZDNET shortcomings

          and a RAID 6 controller costs "insert vendor's extortionate price here".

          So you look at symform which allows you to do everything FOR NO CHARGE, providing you pledge twice the amount of data you want to protect and have a server on all the time (80%+ will do). In other words you have 1TB of your 3TB disk and symform has the rest for other people's stuff.

          At first this might sound like a big ask ... but think about it.

          - how much is it going to cost to have two disks for RAID 1 on ALL the family's machines? One very large disk is far cheaper than several pairs of small disks.

          - RAID 6 sounds very secure ... but symform is 'any 64 locations from a total of 96 at 80+% availability' which is far more 'available' than your buddy being out of town or at a party! Remember the AMAZON outages and the earthquake in Japan? That leaves 94 of 96. Also if you do need an emergency restore, data is coming from many locations and retrieval will max. out your download bandwidth, just like bittorrent!

          - you don't have to talk anybody into it and train them up!

          There is an ideal machine for the task: the HP Microserver. It has very low power consumption , uses ECC memory for quality and it's inexpensive. Although designed for server operating systems ... I've been running symform under Windows 7 and Windows 8 client OS (x64 with 2GB RAM ... it worked with 1GB RAM too!). It shows up the Windows Home Server OS and the previous HP Home Server for the crap it was!! Yes you have initial capital outlay ... but there are no annual subscriptions thereafter.

          PS I don't work for symform or HP.
          jacksonjohn
        • Dear ZDNET

          1. Reinstate the ability to edit posts ... instead of foisting adverts and posts from Timbuktoo on me!
          2 Reinstate HTML editing. Your readers have heard of the Internet you know.
          3. Stop filling the top of my screen with adverts - I kill them before reading on.
          4. Have I mentioned ads. yet?
          5. Get a grip.
          jacksonjohn
    • Accessibilty

      If one wants to have the data accessible and shared across devices and platforms its a good idea to have it on a decentralized network with many nodes. I'm sure there are other reasons but that one is the most obvious that comes to mind. Quite simply though, if you don't have need for a service then don't use it... bu that doesn't mean that others don't have needs for that service.
      techadmin.cc@...
  • I'm not buying into the commercial cloud

    But I won't participate in P2P strategies either.

    My plan is running my own NAS box on my LAN. Another option is attaching storage device(s) to a router, such as Netgear's WNDR3800.

    Currently I have an economical symmetric 18 mb/sec home internet connection, fiber-optic cable all the way to my house. I live just a couple of miles from Google's gigabit fiber installation in the Kansas City area, where the Google network may hopefully expand to my neighborhood or my current fiberoptic provider may easily offer increased speeds at a better price point (if threatened by Google, and the probability is high) as the fiber-optic infrastructure is already in place.

    So my question is: if I have the knowledge and the infrastructure in place to make my own personal cloud, why would I buy storage on somebody else's system(s). All I need do is periodically back up my cloud and store the back ups off-site.

    But I don't care about mobile access to my data. If I need that much data away from home, portable USB drives are dirt cheap right now. Why expose my home LAN or my personal data to the whole internet?

    The cloud isn't necessary for everybody. Therefore the P2P strategies suggested are irrelevant to me too.
    djchandler
  • ok, an example

    who in their right mind dosnt have multiple backups?
    my lowly personal box is backed up in 5 places, 3 of which are completely automated
    2 of them are simply for my convience, SSD to SSD and SSD to RAID 5

    if i lose all my onsite backups it wont matter how easily i can access the data
    any event that took out all my backups will also take out all my other stuff...
    at that point the fact i can access the data from other devices wont matter till i have other devices
    so in order for your system to be of any use i need a standby facility and there is my point, if i have to have another data center in the first place why use the p2p?

    if your comming up with a system that can give access to your customers data to them no matter what, p2p aint going to work in the first place

    the fact your considering p2p means you already accepted major down time for your customer

    if my friend and i are co-locating why use p2p in the first place?
    conobs
  • Cloud

    Best Cloud back up I have used yet. http://goo.gl/nNISh
    rjo66
  • P2P cloud sounds great but it needs De-Dupe

    With everyone saving the same songs and movies, its a waste of resources to back the same file up 10,000 times.
    Reality Bites
  • P2P has actually been a wild success . . .

    "According to the digital cognoscenti, this breed of cloud computing is the way of the future."

    The so-called "digital cognoscenti" that ZDNet usually refers to haven't shown that their way is better. They argue by authority and haughtiness rather than by actual reasoning. From what I've heard so far, there's really nothing inherently superior about the datacenter approach over the P2P approach.

    "But what if there was another way to create a global cloud, one that didn't involve cementing the dominance of global technology companies like Microsoft, Google and their ilk? Would you use it?"

    I already do, actually, in many cases. And actually, so do you. Do you still use email?

    "One would be Wuala, a P2P file storage technology developed by researchers at the University of Zurich that was spun off into its own company and subsequently brought by consumer storage giant Lacie."

    I should note that Dropbox, which supports both datacenter and P2P approaches (LAN sync), is very successful. And any college kid knows about P2P file sharing, and Bittorrent has also been successful.

    P2P file sharing actually saw wild successes. The only problem was that they were usually used for legally questionable music sharing, which gave them a bad rep. But the technology itself is very good, and had they not been so heavy on the illegal usage, they may have stuck around longer and not have been shut down so much.

    "I have nothing against the model, it just hasn't worked out well in the past."

    Sure it has. And it still works out today.

    Current users of P2P technology:

    -Dropbox (LAN sync)
    -Blizzard (The World of Warcraft launcher is actually a Bittorrent client!)
    -Skype
    -The TOR network
    -Email (SMTP relays are peers of each other)
    -One could argue that having multiple data centers is actually a form of P2P (the datacenters are peers of each other)
    -Bittorrent itself is still in business, as are several other file sharing clients.
    -The internet itself!! Internet protocols themselves are inherently P2P, and P2P was really the original intent of the internet. This whole datacenter thing is something we tacked on later. On this basis alone, one could easily argue that P2P has actually been wildly successful, even more so than data centers.
    CobraA1
    • Couple of clarifications

      Hey CobraA1,
      Thanks very much for your detailed reply. Just a quick note to stress - if you weren't already aware - that Dropbox sits directly on Amazon so dropbox-use leads to infrastructure buildout by AWS. As for the broader use of the term P2P - interesting points and I'll keep them in mind for future articles.
      thanks for reading,
      Jack
      Jack Clark
  • And the value added security is where?

    A key component of any IT dependent business is availability. It does not seem to me that a P2P network can maintain any form of viable "availability". And this does not begin to address integrity and confidentiality. But then again, these are all key aspects sacrificed in Cloud computing as well... imagine a small "Mom and Pop" business using a mega-million conglomerate IT company and suffering outages, file loss or security breaches... do you think they would have the financial resources to take those Cloud corps to court for breach of contract? And yes, this would apply for any P2P storage solution as well.
    tomxfoolery