University scientists set new data transfer records at SC06

Maybe it wasn't NASCAR, but at SuperComputing 06, scientists were pouring on the heat to see who could set the data transfer rate record at the "Bandwidth Challenge," reports Campus Technology

A joint team of computer scientists, network engineers and physicists from the California Institute of Technology, CERN and the University of Michigan, together with partners at the University of Florida and Vanderbilt, as well as participants from Brazil and Korea, set off to leave old data transer records in the dust.

The team achieved a peak throughput of 17.77 gigabits per second (Gbps) between clusters of servers on the show floor and CalTech in Pasadena.

Using the rules set for the challenge, the researchers used a single 10-Gbps link provided by National Lambda Rail that carried data in both directions.

The main innovation during the demonstration was the Fast Data Transport (FDT), a Java application developed by Iosif Legrand of Caltech, that runs on all major platforms and achieves stable disk reads-and-writes and smooth data flow across a long-range network. FDT streams a large set of files across an open TCP socket, so that a typically large data set composed of thousands of files can be sent or received at full speed without the network transfer restarting between files.