Hardware-based rootkit detection proven unreliable

Summary:For years, we've been convinced by companies like Komoku and BBN Technologies that hardware-based RAM acquisition is the most reliable and secure way to sniff out the presence of a sophisticated rootkit on a compromised machine. Not so fast, says Joanna Rutkowska, a security researcher at COSEINC Malware Labs.

For years, we've been convinced by companies like Komoku and BBN Technologies that hardware-based RAM acquisition is the most reliable and secure way to sniff out the presence of a sophisticated rootkit on a compromised machine.

Joanna Rutkowska, Jamie Butler
Not so fast, says Joanna Rutkowska, a security researcher at COSEINC Malware Labs.

Rutkowska, an elite hacker who specializes in offensive rootkit research, has found several ways to manipulate the results given to hardware-based solutions (PCI cards or FireWire bus).

At this year's Black Hat DC conference, Rutkowska demonstrated three different attacks against AMD64 based systems, showing how the image of volatile memory (RAM) can be made different from the real contents of the physical memory as seen by the CPU.

Rutkowska's research, though purely theoretical, underscores the need for multiple solutions (hardware and software) to work in tandem during forensics. It also highlights just how scary the threat from sophisticated rootkits can be. If, as Rutkowska proved, forensic examiners cannot rely on images collected from RAM, then it's basically game over.

Jamie Butler, a rootkit guru who works with software- and hardware-based anti-rootkit tools, said he was "very impressed" with Rutkowska's presentation. "We already know that software isn't reliable and now we know that you really can't trust the hardware either. You really need to combine both and, even then, you just never know," Butler said.


 
  Black Hat Gallery:Hackers discuss weaknesses in Wi-Fi drivers, RFID proximity devices and hardware-based forensics. Images in our gallery.  

 

"I really don't want to meet the attacker who is at that level," he said. "That is scary stuff," Butler said, referring to the techniques used during Rutkowska's presentation.

In three different scenarios, Rutkowska showed how an attacker can crash a machine during memory acquisition. In this case, it would be a denial-of-service against the forensics examiner looking to find traces of malware on a hijacked machine.

She also described a "covering attack" where the malware is programmed to present garbage data to the hardware trying to read physical memory.

A third scenario is what Rutkowska described as a "full replacing attack" where the malware author not only hides malicious code from the memory acquisition tool but actually provides arbitrary/fake content to the examiner.

The overall problem, Rutkowska explained, is the design of the system that makes it impossible to reliably read memory from computers. "Maybe we should rethink the design of our computer systems so they they are somehow verifiable," she said.

Rutkowska suggests that hardware vendors come up with a special "auditing" interface dedicated only to memory acquisition.

"I'm thinking about motherboard manufacturers adding a special port which would allow for *direct* (this time really "direct") access to RAM and potentially some other critical resources like e.g. CPU system registers and maybe even caches," she said.

Here are the slides from Rutkowska's presentation (PDF). 

Topics: Hardware

About

Ryan Naraine is a journalist and social media enthusiast specializing in Internet and computer security issues. He is currently security evangelist at Kaspersky Lab, an anti-malware company with operations around the globe. He is taking a leadership role in developing the company's online community initiative around secure content managem... Full Bio

zdnet_core.socialButton.googleLabel Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.