Earlier this month I had the opportunity to present RE:Trace at the Recon conference, a reverse engineering conference held every other year in Montreal, Canada. The conference consisted of three days of training and three days of talks in a single track. Topics include reversing polymorphic malware, overcoming code obfuscation and anti-debugging techniques, local privilege escalation through the ALPC kernel interface on Vista, and hacking OS X. The technical level of the conference was excellent and everything was very well organized from the nightly events to the downloadable presentation videos on the conference network.
There were so many interesting talks, but since I cannot cover them all, here are some highlights:
Gerardo Richarte presented two small tools to assist in reverse engineering code. One of these tools was more of a methodology and he called it iterative decompilation. The idea is to reverse parts of a program into high level code one piece at a time. He would first examine a function's behavior in assembly and then re-implement the function in C. By compiling this C into a DLL and patching it into the process at runtime he could replace the original code with his own implementation and test to ensure it had the same behavior. Following this processes he can reverse the important parts of an application in a progressive fashion.
Craig Smith from Neohapsis presented on using virtual machine to implement code obfuscation. Don't think VMWare virtual machines here. Instead think custom instruction sets and a runtime interpreter. He covered the basics of this technique which were popularized by honeynet SOTM32 challenge. The idea is to create a custom instruction set and then use it to implement the logic you want obfuscated. You can apply this approach to hide system calls or important calculations. The bottom line is that a small effort on the developers part can create a lot more work for the reverser. More details and his sample code can be found on the Neohapsis blog.
Isn't Netware dead? Well, pretty much, but Nicolas Pouvesle ran into it enough times (twice) that he decided to explore it, and by explore it I mean serious kernel exploit fu. One of his biggest challenges was coming up with a payload that was generic enough to work across the many versions of Netware. He could not do a simple connect back shell, because in Netware there is no system level user, which means that there is no shell! There is a system console, however interfacing with it involves converting between bitmaps and character buffers and injecting keystrokes. This approach still did not work on all Netware version, so Pouvesle found a way to create a user within the LDAP database that Netware uses for administrative web interface access. Creating LDAP users could only be achieved with library calls and not kernel calls so he jumped through more hoops to get visibility of those libraries from the kernel and resolve their encrypted symbols. His slides from slideshare can be found here.
Overall this was an excellent conference and I recommend it for anyone involved in reverse engineering. For those of you interested, more information is posted on the Recon website.
* Tiller Beauchamp is an information security professional and noted researcher. He is perhaps best known in the security and Mac communities for creating the RE:Trace framework, a Ruby-based framework for interacting with the powerful DTRace tool for vulnerability research. He blogs with fellow researcher and co-creator of the RE:Trace framework at the POP/POP/RET blog and can also be seen at the upcoming Black Hat Las Vegas 2008 conference.