Vista's ASLR not so random, but does it matter?
On the heels of its exposé of weaknesses in the UAC (user account control) mechanism, Symantec rolled out a Vista security portal with three new research papers discussing legacy threats that affect the brand new operating system.
The three papers provide a technical overview of Redmond's implementation of ALSR (address space layout randomization), the GS stack protection in Vista and the operating system's resilience to today's malware variants.
Symantec researcher Ollie Whitehouse is on the Black Hat schedule with a talk on ALSR, which is on-by-default in Vista to thwart memory manipulation attacks.
In a nutshell, Whitehouse found that Microsoft's implementation of ASLR isn't 100 percent effective against automated malware attacks that rely on predicting the memory layouts of loaded programs.
Our research also shows that applications that leverage the Microsoft HeapAlloc() function are not afforded the same level of protection as those that leverage the ANSI C heap allocation API malloc(). As a result, third-party software that explicitly uses Microsoft’s API is potentially more vulnerable to exploitation than software that does not. Also apparent is that using CreateHeap() followed by HeapAlloc() improves the entropy slightly over using malloc() alone. Finally, results show fewer consecutive duplicates than expected in the PEB randomization. This result adds to the evidence that the source of entropy used within ASLR is poorly used.
Although Whitehouse's findings are sure to set tongues wagging in the security research community, it should be noted that ASLR is just one of many core enhancements that work together to secure Vista.
Beyond ASLR, there is /GS, a compile-time option in Visual C++ that adds stack-based buffer overrun detection, /SafeSEH, Data Execution Protection and Function Pointer Obfuscation.
As Microsoft's Michael Howard explained when ASLR was added to Windows Vista Beta 2, it is not a panacea or a replacement for insecure code. "[But] when used in conjunction with other technologies,it is a useful defense because it makes Windows systems look 'different' to malware, making automated attacks harder."