Debugging Crashes in Windows Applications: The Null Pointer Dereference

Windows C++ developers remain all too familiar with the standard Windows crash dialog. This post is an attempt to teach developers how to understand the data the crash dialog reports to diagnose difficult issues. A basic understanding of assembly language is assumed; for more background on these topics please read Matt Pietrek’s “Under The Hood” articles in the Microsoft Systems Journal February 1998 and June 1998 issues.

To begin with, let’s write an application that crashes with a null pointer dereference:

// DerefNullPointer.c: Crash by dereferencing the null pointer.

static void DoCrash()
{
    *((char*) 0) = 1;
}

int main(int argc, char* argv[])
{
    DoCrash();
    return 0;
}

I placed the null pointer dereference in its own function for reasons that will later become clear. Now, let’s compile the application from the command line:

C:\Proj\Crashes> cl /Od /nologo /c DerefNullPtr.c
C:\Proj\Crashes> link /SUBSYSTEM:CONSOLE /OUT:DerefNullPtr.exe DerefNullPtr.obj

Now run the application:

C:\Proj\Crashes> DerefNullPtr.exe

The standard Windows crash dialog pops up.

Normally users will click “Send Error Report” or “Don’t Send”. If you have access to the machine and a debugger installed you can click “Debug”. Alternatively, you can view the error report details right away by clicking “click here.” If you do, an error report dialog will pop up.

The most important piece of information from the error report dialog is the offset of the instruction which caused the crash. Note that this value is an offset — it’s the address of the crashing instruction relative to where DerefNullPtr.exe is loaded into memory.

We now have enough information to see the instruction which is causing the crash. One way to do this is to use Doron Holan’s windbg1 dump file trick:

C:\Proj\Crashes> windbg -z DerefNullPtr.exe

Microsoft (R) Windows Debugger  Version 6.6.0003.5
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Proj\Crashes\DerefNullPtr.exe]
Symbol search path is: SRV*C:\websymbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
ModLoad: 00400000 00408000   C:\Proj\Crashes\DerefNullPtr.exe
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000000
eip=00401041 esp=00000000 ebp=00000000 iopl=0         nv up di pl nz na pe nc
cs=0000  ss=0000  ds=0000  es=0000  fs=0000  gs=0000             efl=00000000
*** WARNING: Unable to verify checksum for DerefNullPtr.exe
*** ERROR: Module load completed but symbols could not be loaded for DerefNullPtr.exe
DerefNullPtr+0x1041:
00401041 6a18             push    0x18
0:000> u DerefNullPtr+0×1013
DerefNullPtr+0×1013:
00401013 c6050000000001   mov     byte ptr [00000000],0×1
0040101a 5d               pop     ebp
0040101b c3               ret
0040101c 833dc872400002   cmp  dword ptr [DerefNullPtr+0×72c8 (004072c8)],0×2
00401023 7405             jz      DerefNullPtr+0×102a (0040102a)
00401025 e8f0040000       call    DerefNullPtr+0×151a (0040151a)
0040102a ff742404         push    dword ptr [esp+0×4]
0040102e e870030000       call    DerefNullPtr+0×13a3 (004013a3)

The crashing instruction is mov byte ptr [00000000],0x1. Clearly the bug is a null pointer dereference.

Had we produced a map file at link time, we could have done even better. Let me show you:

C:\Proj\Crashes> link /SUBSYSTEM:CONSOLE /MAP:DerefNullPtr.map /OUT:DerefNullPtr.exe DerefNullPtr.obj

The map file looks like:

DerefNullPtr

Timestamp is 462fae49 (Wed Apr 25 14:38:49 2007)

Preferred load address is 00400000

Start         Length     Name                   Class
0001:00000000 00003b68H .text                   CODE
0002:00000000 000000c4H .idata$5                DATA
...
0003:00000030 00000274H .data                   DATA
0003:000002c0 00000598H .bss                    DATA

Address         Publics by Value              Rva+Base     Lib:Object

0000:00000000       __except_list              00000000     <absolute>
0000:00000002       ___safe_se_handler_count   00000002     <absolute>
0001:00000000       _main                      00401000 f   DerefNullPtr.obj
0001:0000001c       __amsg_exit                0040101c f   LIBC:crt0.obj
0001:00000041       _mainCRTStartup            00401041 f   LIBC:crt0.obj
...
0003:00000850       __FPinit                   00407850     <common>
0003:00000854       __acmdln                   00407854     <common>

entry point at        0001:00000041

Static symbols

0001:00000010       _DoCrash                   00401010 f   DerefNullPtr.obj
0001:000002a2       _doexit                    004012a2 f   LIBC:crt0dat.obj
0001:0000078b       _parse_cmdline             0040178b f   LIBC:stdargv.obj

The first thing to note is that all addresses in the map file are relative to the preferred load address, 0x00400000. Therefore, we’re looking for the function which contains the address 0x00400000 + 0x00001013 = 0x00401013. To do so, we should look for the function which has the largest address less than or equal to the address of the crash. The function _DoCrash() is at the address 0x00401010, so the crash appears to be at the address _DoCrash+0x3.

Better still is to create a PDB file alongside your retail builds, as in:

C:\Proj\Crashes> cl /Zi /Od /nologo /c DerefNullPtr.c
C:\Proj\Crashes> link /DEBUG /SUBSYSTEM:CONSOLE /PDB:DerefNullPtr.pdb /MAP:DerefNullPtr.map /OUT:DerefNullPtr.exe DerefNullPtr.obj

Note that the /DEBUG option may be a little confusing — it really should named /CREATEPDB.

Unfortunately, this change will invalidate the previous crash address. (This wouldn’t be a problem if building retail PDBs was part of your normal process) However, we can reproduce the bug within the debugger:

C:\Proj\Crashes> windbg DerefNullPtr.exe

Microsoft (R) Windows Debugger  Version 6.6.0003.5
Copyright (c) Microsoft Corporation. All rights reserved.

CommandLine: DerefNullPtr.exe
Symbol search path is: SRV*C:\websymbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
ModLoad: 00400000 0040d000   DerefNullPtr.exe
ModLoad: 7c900000 7c9b0000   ntdll.dll
ModLoad: 7c800000 7c8f4000   C:\WINDOWS\system32\kernel32.dll
(3e4.7f4): Break instruction exception - code 80000003 (first chance)
eax=00241eb4 ebx=7ffd9000 ecx=00000000 edx=00000001 esi=00241f48 edi=00241eb4
eip=7c901230 esp=0012fb20 ebp=0012fc94 iopl=0         nv up ei pl nz na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000202
ntdll!DbgBreakPoint:
7c901230 cc               int     3
0:000> g
(3e4.7f4): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=003214a8 ebx=7ffd9000 ecx=00000001 edx=7c90eb94 esi=00000a28 edi=00000000
eip=00401023 esp=0012fedc ebp=0012fedc iopl=0         nv up ei pl zr na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00010246
*** WARNING: Unable to verify checksum for DerefNullPtr.exe
DerefNullPtr!DoCrash+0×3:
00401023 c6050000000001   mov     byte ptr [00000000],0×1 ds:0023:00000000=??
0:000> kb
ChildEBP RetAddr  Args to Child
0012fedc 00401018 0012ffc0 00401245 00000001 DerefNullPtr!DoCrash+0×3 [c:\proj\crashes\derefnullptr.c @ 3]
0012fee4 00401245 00000001 00321470 003214a8 DerefNullPtr!main+0×8 [c:\proj\crashes\derefnullptr.c @ 9]
0012ffc0 7c816fd7 00090000 00a7fa9c 7ffd9000 DerefNullPtr!mainCRTStartup+0×173 [f:\vs70builds\3077\vc\crtbld\crt\src\crt0.c @ 259]
0012fff0 00000000 004010d2 00000000 78746341 kernel32!BaseProcessStart+0×23

Using PDBs, you have the ability to map instruction addresses to a file and line number, even with retail builds. I recommend producing PDBs as part of your build process and placing them somewhere accessible by all developers on your team.

1. Windbg is an extremely powerful debugger included with Microsoft’s free Debugging Tools for Windows.

Advertisements