Data Expedition accelerating "big data"

Seth Noble, CEO of Data Expedition, Inc. (DEI), reached out to me after I posted Aspera - High speed data transfers and EMC and Aspera teaming up to handle "big data". He was concerned that I had not learned about his company, Data Expedition, or its "Multipurpose Transaction Protocol" (MTP/IP). He was right.

The problem DEI is addressing

As with Aspera, Data Expedition's goal is using new, sophisticated software technology to drastically accelerated file transfers that are currently being handled by the file transfer protocol (FTP), the secure file transfer protocol (SFTP), the hypertext transfer protocol (HTTP) or the common internet file system (CIFS).

Most of these older protocols were designed when computers in the same computer room or datacenter needed to communicate with one another. HTTP, on the other hand, was really designed to deliver relatively short HTML-encoded Web pages and does not function quite as effectively when used to deliver very large data files over noisy networks.

So, some of the basic assumptions that are at the foundation of the design of these protocols no longer really apply. So, file transfer reliability and performance are not up to the requirements of today's applications.

What DEI offers

Here's how DEI descibes their offerings:

MTP™/IP Applications

Software built with Multipurpose Transaction Protocol® technology is able to achieve maximum performance and reliability across a broad and growing range of network environments. DEI offers a number of MTP/IP products, including software development kits that let you bring scalable networking to your application.

ExpeDat™

This drop in replacement for conventional FTP applications delivers maximum network performance and security in familiar interfaces. Create and send self-contained droplets for one-step, drag-and-drop file transmission. Pre-configurable GUI clients give administrators complete control.

SyncDat™

SyncDat compares the contents of a local folder with a remote one and automatically transfers changed or missing files. This combines the raw speed of MTP/IP with automated, selective copying. Applications include scheduled data deployments, file backups, and multi-system synchronization.

HyperGate™

HyperGate merges the power of MTP/IP technology with the ubiquity of web browsers and servers. By accelerating document transfers between standard web browsers and servers, HyperGate gives you improved performance while preserving familiar web interfaces and web server infrastructure.

DEI also offers a number of development kits to assist its customers in embedding the technology into their own applications or to create their own client software.

Snapshot analysis

Both DEI and Aspera believe that the exploding data requirements of most large companies and digital content suppliers need help to reduce the time it takes to deliver large amounts of information. DEI also thinks that their approach to offering their technology, that is offering tool kits to embed the technology in the customers' own applications, is more useful to its clients.

I think that both companies are addressing a real need, but a need that is currently felt more by very large companies and suppliers of storage virtualization technology, database technology and digital content than the market as a whole.

That being said, if your organization falls into one of those groups and it has a need to accelerate the transfer of "big data" files, DEI should be evaluated.