Rutkowska faces '100% undetectable malware' challenge
At last year's Black Hat security conference, stealth malware researcher Joanna Rutkowska caused a stir with the introduction of Blue Pill, a new technology she claims can create malware that remains "100 percent undetectable."
The challenge is being laid out by Thomas Ptacek (left), co-founder of Matasano Security, Nate Lawson of Root Labs and Symantec's Peter Ferrie -- three high-profile researchers out to prove that virtual machine rootkits (malicious hypervisors) are actually easier to detect than normal rootkits.
The challenge will closely resemble the CanSecWest MacBook takeover contest won by Dino Dai Zovi -- two untouched laptops of the make/model of Rutkowska's choosing will be provided for her to plant Blue Pill on one.
"She picks one in secret, installs her kit, sets them up however she wants," Lawson explained in anLawson said there are no fine-print or caveats. "Our goal is to make the ground rules as simple as possible and in Rutkowska's (right) favor as much as possible, given that we think a 100% undetectable rootkit is impossible," he declared.
"If she has any particular requests, we'll almost certainly grant them," he added.
Lawson, who previously worked at Cryptography Research where he co-designed the Blu-ray content protection layer (BD+) , is adamant that hypervisor rootkits like Blue Pill and Dai Zovi's Vitriol can only infect a machine in two ways.
[ SEE: Rutkowska launches Invisible Things Labs ]
The first path is for the attacker to try to leave as much as possible unmodified, which is a non-starter. "For example, not virtualizing the CPU clock cycle counter (TSC) means the detector can see the stolen cycles that the rootkit uses," Lawson argues.
The second path, which is used by Rutkowka, is to try to hook everything and emulate it perfectly. This, the three researchers will argue at Black Hat, is simply not feasible.
Lawson's argument:
To perfectly emulate the unmodified system, the rootkit must emulate by not only "fixing up" values like the TSC, but it must fully support all functionality of the unmodified system, including all bugs and performance.
For example, if the system supports VT virtualization, the rootkit must implement this also. That means the OS needs to be able to launch its own hypervisor even though the rootkit is already running as a hypervisor.
Even if all that is accomplished perfectly, Lawson says the rootkit author faces the impossible task of needing to emulate all bugs and quirks of the original system.
[ SEE: Hardware-based rootkit detection proven unreliable ]
"The crux of the matter is that a perfect emulator of any sufficiently complex system would have to be a bug-free program, and we don't know how to write those yet," he argued. "The important thing to consider when writing a rootkit is what layer to implement it at. Joanna chose "entire x86 PC", which we argue is too big a cross-section."
Matasano's Ptacek, who has spent a lot of time studying Rutkowska's work, said the challenge team will compare the behavior of the system to known norms to find the presence of Blue Pill.
Earlier this year, Rutkowska presented new research at Black Hat DC to show how physical memory acquisition can be cheated to avoid rootkit detection. She demonstrated three different attacks against AMD64 based systems, showing how the image of volatile memory (RAM) can be made different from the real contents of the physical memory as seen by the CPU.
This is believed to be an advancement of the Blue Pill concept but Lawson thinks this simply increases the rootkit's surface area and makes it easier to detect.
"I think the best rootkit is the simplest," Lawson added.
I e-mailed Rutkowska for a comment and will update this entry as necessary. Rutkowska has responded with a list of ground rules, including a financial demand that has scuttled any plans for a Black Hat face off.