NSA leak exposes Red Disk, the Army's failed intelligence system
The contents of a highly sensitive hard drive belonging to a division of the National Security Agency have been left online.
The virtual disk image contains over 100 gigabytes of data from an Army intelligence project, codenamed "Red Disk." The disk image belongs to the US Army's Intelligence and Security Command, known as INSCOM, a division of both the Army and the NSA.
Security
The disk image was left on an unlisted but public Amazon Web Services storage server, without a password, open for anyone to download. Unprotected storage buckets have become a recurring theme in recent data leaks and exposures. In the past year alone, Accenture, Verizon, and Viacom, and several government departments, were all dinged by unsecured data.
Chris Vickery, director of cyber risk research at security firm UpGuard, found the data and informed the government of the breach in October. The storage server was subsequently secured, though its owner remains unknown.
The leak marks yet another exposure of classified government data. Since the Edward Snowden disclosures in 2013, the agency made headlines last year when Harold Martin, an NSA contractor, was indicted for removing terabytes of secret data from the agency's headquarters. Another contractor, Reality Winner, was indicted this year for leaking classified secrets to news site The Intercept.
When approached prior to publication, an NSA spokesperson did not return a request for comment. An INSCOM spokesperson was unable to comment by the time of publication.
The disk image, when unpacked and loaded, is a snapshot of a hard drive dating back to May 2013 from a Linux-based server that forms part of a cloud-based intelligence sharing system, known as Red Disk. The project, developed by INSCOM's Futures Directorate, was slated to complement the Army's so-called distributed common ground system (DCGS), a legacy platform for processing and sharing intelligence, surveillance, and reconnaissance information.
Each branch of the military has its own version of the intelligence sharing platform -- the Army's is said to be the largest -- but the Army's system struggled to scale to the number of troops who need it.
Red Disk was envisioned as a highly customizable cloud system that could meet the demands of large, complex military operations. The hope was that Red Disk could provide a consistent picture from the Pentagon to deployed soldiers in the Afghan battlefield, including satellite images and video feeds from drones trained on terrorists and enemy fighters, according to a Foreign Policy report.
But the system was slow, crash prone, and difficult to use. A memo from 2014 by soldiers with one deployed brigade said the system was "a major hindrance to operations," as reported by the Associated Press.
The Pentagon spent at least $93 million on Red Disk, but it was never fully deployed in the field. The project has since been largely seen as a failure.
While the contents of the disk are readable, the system itself wouldn't boot -- likely because it relies on dependent systems and servers that are only available from within the Pentagon's network. But the files alone offer a glimpse into how Red Disk worked.
Red Disk was a modular, customizable, and scalable system for sharing intelligence across the battlefield, like electronic intercepts, drone footage and satellite imagery, and classified reports, for troops to access with laptops and tablets on the battlefield. Marking files found in several directories imply the disk is "top secret," and restricted from being shared to foreign intelligence partners.
Red Disk could draw in vast amounts of intelligence, documents, videos, and audio from several sources, including signals intelligence, radar, wide area aerial surveillance, drones, and audio databases -- some fed in directly from the NSA. That raw, mostly unstructured data passed through software called NiFi (formerly NiagraFiles), a since declassified NSA system to support highly scalable and flexible data flows, which directs different kinds of data across multiple computer networks and geographically dispersed sites. That was particularly useful for Red Disk, which relied on obtaining and sending data over wide areas.
An icon found in the leaked files, used to "target" individuals of interest. (Image: supplied)
The data then was sorted and organized through various filters. The data would be indexed, allowing analysts to carry out metadata tagging, extract geo-temporal information, and run a data provenance process to verify the source and owner of certain data.
All the collected intelligence would be stored in a central repository to be analyzed, correlated, and enriched. An analyst could pull intelligence from the repository based on their security clearance. An analyst would obtain their access from their Pentagon-issued certificate-based credentials, which grants them access only to data they are permitted to see.
The system also comes with several plug-in apps, allowing analysts to interact with intelligence data. One program includes DOMEX, a document and media program for analyzing seized documents and electronic evidence.
Several files also point to biometric analysis tools, and an integration of human language technologies to allow analysts to query reports and play audio in English.
One image found on the drive reveals how analysts can target individuals of interest, such as potential terrorists, in the DCGS system for later action -- such as by ground troops or autonomous drones.
Vickery noted that the disk image also contains other sensitive files, including private keys used for the system to access other servers on the intelligence community's network. The keys belong to a third-party firm, Invertix, a working partner of INSCOM and a key developer of Red Disk.
Invertix, now named Altamira Technologies, did not respond to a request for comment.
INSCOM's data exposure is the latest in a long list of government leaks in the past year.
Several government agencies, including US Central Command and US Pacific Command and the National Geospatial Intelligence Agency, charged with analyzing top secret satellite imagery, have admitted exposing sensitive or classified information.
Vickery, who searches for exposed data online, has been responsible for finding much of the data. But he said the latest data exposure was entirely avoidable.
"What are we doing wrong when 'top secret' data is literally two mouse clicks away from worldwide exposure?" he said. "How did we get here, and how do we find a way out?"