Using diStorm with Python 2.6 and Python 3.x, revisited

In a previous post, we’ve seen how to wrap the diStorm disassembler library in Python, using ctypes. This still left us with the task of building the dynamic link library for our platform and installing it manually, which is not as easy as it may seem – among other small problems you may find, the new versions of Visual Studio try to force the use of the latest C++ runtime redistributables, which may not be present in most Windows installations.

Today, I’m introducing a new ctypes wrapper for diStorm, this time with all binaries prebuilt and packaged together. The installer script automatically detects the target platform and installs the right binary. It comes with the following prebuilt binaries:

• Windows on x86 and AMD64 processors
• Linux on x86 and AMD64 processors (built using Ubuntu, but should work in other distros)
• Mac OS X on x86 and PowerPC processors (untested, I don’t have a Mac to play with yet)

Since the installer code is pretty much generic, it should be easy to add new platforms by simply creating the corresponding subdirectory and placing the python code and prebuilt binary in it. Contributions are welcome! 🙂

Download

Python 2.x

• Source code (all platforms): distorm-1.7.30.zip
• Source code (all platforms): distorm-1.7.30.tar.gz
• Windows 32 bits installer: distorm-1.7.30.win32.exe
• Windows 32 bits MSI installer: distorm-1.7.30.win32.msi
• Windows 64 bits installer: distorm-1.7.30.win-amd64.exe
• Windows 64 bits MSI installer: distorm-1.7.30.win-amd64.msi

Python 3.x

• Source code (all platforms): distorm-1.7.30.zip
• Source code (all platforms): distorm-1.7.30.tar.gz
• Windows 32 bits installer: distorm-1.7.30.win32.exe
• Windows 32 bits MSI installer: distorm-1.7.30.win32.msi
• Windows 64 bits installer: distorm-1.7.30.win-amd64.exe
• Windows 64 bits MSI installer: distorm-1.7.30.win-amd64.msi

DBG_CONTINUE vs. DBG_EXCEPTION_HANDLED

Something odd just happened to me while trying out my debugger on Windows XP 64 bits – whenever I stepped on or traced an instruction, the instruction pointer would go back to the same instruction the first time. The second time, it worked. So I had to step twice on each instruction to debug a program.

This was clearly wrong, but I couldn’t see anything in my code that would produce that. Furthermore, in 32 bits the exact same code was working alright!

After scratching my head for a while, I went to MSDN in search of enlightment, and found this info at the help page for ContinueDebugEvent:

DBG_CONTINUE (0x00010002L): If the thread specified by the dwThreadId parameter previously reported an EXCEPTION_DEBUG_EVENT debugging event, the function stops all exception processing and continues the thread. For any other debugging event, this flag simply continues the thread.

DBG_EXCEPTION_NOT_HANDLED (0x80010001L): If the thread specified by dwThreadId previously reported an EXCEPTION_DEBUG_EVENT debugging event, the function continues exception processing. If this is a first-chance exception event, the search and dispatch logic of the structured exception handler is used; otherwise, the process is terminated. For any other debugging event, this flag simply continues the thread.

But I remembered seeing a certain DBG_EXCEPTION_HANDLED value when porting SDK constants and structures to my Win32 API wrappers. So I looked it up and it’s value turned out to be 0x00010001L, which was exactly DBG_CONTINUE minus one, so it wasn’t not just an alias but really a different value, possibly with a different meaning.

So I tried replacing DBG_CONTINUE by DBG_EXCEPTION_HANDLED when handling exceptions in my code and, what do you know, single steping and tracing began to work again!

Apparently, when handling exception events, 32 bits Windows treats DBG_CONTINUE as an alias of DBG_EXCEPTION_HANDLED, while in 64 bits Windows it’s an alias of DBG_EXCEPTION_NOT_HANDLED. This means we can’t possibly write a debugger for 64 bits without using this undocumented value! 😦

Luckily for us, DBG_EXCEPTION_HANDLED seems to work well in 32 bits Windows, at least with XP (didn’t test Windows 2000 yet), so the same code may be used for all versions of Windows. It also works with current versions of Wine, judging from this commit from 2005.

Speaking of Wine, browsing through the sources I found several other “undocumented” status codes:

• DBG_TERMINATE_PROCESS
• DBG_TERMINATE_THREAD
• DBG_CONTROL_C
• DBG_CONTROL_BREAK
• DBG_COMMAND_EXCEPTION

However the last three don’t seem to be supported by Windows XP, as shown in the following disassembly of NTOSKRNL.EXE:

That still leaves us with two new undocumented status codes to play with 🙂