Machine code

It has been a while since I did anything in machine language. Until this past weekend, that is, when I spent a fair bit of time starting at disassembled code with a debugger.

Last week, I upgraded my Sony Ericsson smartphone to the latest version of its operating system. The upgrade process failed on my main computer when, after updated USB drivers for the phone were downloaded, they failed to install. The problem was not specific to the phone: all driver installations failed, with a not very informative code (0xC0000142, which just means that the application failed to initialize.)

Using the very helpful ProcMon utility from sysinternals (now owned by Microsoft) I managed to identify that it was a process named drvinst.exe that failed. This process is invoked automatically by the system every time a USB device is inserted, and also during device driver installations. So why did it fail?

I downloaded the latest Windows debugger (windbg.exe) from Microsoft; this debugger allows me to do things like debug child processes spawned by a parent process. (I later learned that drvinst.exe actually has a feature whereas it waits for a debugger after startup, to help with driver installation debugging; but chances are that I would not have been able to make much use of this feature, as the failure occurred before drvinst.exe actually started to execute its own code.) I attached the debugger to the DCOM service process (which is the one that spawns copies of drvinst.exe.) I was able to determine that it was during the initial process setup stage that this process failed, when it was attempting to attach to the gdi32.dll system library.

I still have no idea why this happens. But with the help of the debugger, I was able to tinker with this process, changing a processor register’s value at just the right spot, allowing it to continue. This inconvenient but workable process allowed me to install drivers for my phone and also updated drivers for my wireless mouse from Microsoft Update.

Perhaps the most incomprehensible bit is that the same thing works fine on an essentially identical computer. The actual failure occurs inside a kernel subroutine (undocumented system call 123Ah, called from GdiDllInitialize in gdi32.dll) that I cannot debug without a kernel debugger (and since I am trying not to mess my machine up too much, I opted not to do kernel debugging). That subroutine does not appear to be doing anything particularly magical. I checked and all relevant files and Registry settings are identical on the two machines. So it remains a mystery for now… nonetheless, it was educational. I learned a lot about driver installation in Windows 7, about process startup, and incidentally, about the ReactOS project whose open source equivalents to the relevant system components helped me a great deal to understand what was going on.