Debugging Crashes in Windows Applications: The Null Pointer Dereference

Windows C++ developers remain all too familiar with the standard Windows crash dialog. This post is an attempt to teach developers how to understand the data the crash dialog reports to diagnose difficult issues. A basic understanding of assembly language is assumed; for more background on these topics please read Matt Pietrek’s “Under The Hood” articles in the Microsoft Systems Journal February 1998 and June 1998 issues.

To begin with, let’s write an application that crashes with a null pointer dereference:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


  
// DerefNullPointer.c: Crash by dereferencing the null pointer.

static void DoCrash()
{
    *((char*) 0) = 1;
}

int main(int argc, char* argv[])
{
    DoCrash();
    return 0;
}

I placed the null pointer dereference in its own function for reasons that will later become clear. Now, let’s compile the application from the command line:

1
2


C:\Proj\Crashes> cl /Od /nologo /c DerefNullPtr.c
C:\Proj\Crashes> link /SUBSYSTEM:CONSOLE /OUT:DerefNullPtr.exe DerefNullPtr.obj

Now run the application:

1

C:\Proj\Crashes> DerefNullPtr.exe

The standard Windows crash dialog pops up.

Normally users will click “Send Error Report” or “Don’t Send”. If you have access to the machine and a debugger installed you can click “Debug”. Alternatively, you can view the error report details right away by clicking “click here.” If you do, an error report dialog will pop up.

The most important piece of information from the error report dialog is the offset of the instruction which caused the crash. Note that this value is an offset — it’s the address of the crashing instruction relative to where DerefNullPtr.exe is loaded into memory.

We now have enough information to see the instruction which is causing the crash. One way to do this is to use Doron Holan’s windbg1 dump file trick:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


C:\Proj\Crashes> windbg -z DerefNullPtr.exe

Microsoft (R) Windows Debugger Version 6.6.0003.5
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Proj\Crashes\DerefNullPtr.exe]
Symbol search path is: SRV\*C:\websymbols\*http://msdl.microsoft.com/download/symbols
Executable search path is:
ModLoad: 00400000 00408000 C:\Proj\Crashes\DerefNullPtr.exe
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=00000000
eip=00401041 esp=00000000 ebp=00000000 iopl=0 nv up di pl nz na pe nc
cs=0000 ss=0000 ds=0000 es=0000 fs=0000 gs=0000 efl=00000000
\*** WARNING: Unable to verify checksum for DerefNullPtr.exe
\*** ERROR: Module load completed but symbols could not be loaded for DerefNullPtr.exe
DerefNullPtr+0x1041:
00401041 6a18 push 0x18
0:000> u DerefNullPtr+0×1013
DerefNullPtr+0×1013:
00401013 c6050000000001 mov byte ptr [00000000],0×1
0040101a 5d pop ebp
0040101b c3 ret
0040101c 833dc872400002 cmp dword ptr [DerefNullPtr+0×72c8 (004072c8)],0×2
00401023 7405 jz DerefNullPtr+0×102a (0040102a)
00401025 e8f0040000 call DerefNullPtr+0×151a (0040151a)
0040102a ff742404 push dword ptr [esp+0×4]
0040102e e870030000 call DerefNullPtr+0×13a3 (004013a3)

The crashing instruction is mov byte ptr [00000000],0x1. Clearly the bug is a null pointer dereference.

Had we produced a map file at link time, we could have done even better. Let me show you:

1

C:\Proj\Crashes> link /SUBSYSTEM:CONSOLE /MAP:DerefNullPtr.map /OUT:DerefNullPtr.exe DerefNullPtr.obj

The map file looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


DerefNullPtr
Timestamp is 462fae49 (Wed Apr 25 14:38:49 2007)
Preferred load address is 00400000
Start Length Name Class
0001:00000000 00003b68H .text CODE
0002:00000000 000000c4H .idata$5 DATA
...
0003:00000030 00000274H .data DATA
0003:000002c0 00000598H .bss DATA

Address Publics by Value Rva+Base Lib:Object
0000:00000000 __except_list 00000000 <absolute>
0000:00000002 ___safe_se_handler_count 00000002 <absolute>
0001:00000000 _main 00401000 f DerefNullPtr.obj
0001:0000001c __amsg_exit 0040101c f LIBC:crt0.obj
0001:00000041 _mainCRTStartup 00401041 f LIBC:crt0.obj
...
0003:00000850 __FPinit 00407850 <common>
0003:00000854 __acmdln 00407854 <common>
entry point at 0001:00000041

Static symbols
0001:00000010 _DoCrash 00401010 f DerefNullPtr.obj
0001:000002a2 _doexit 004012a2 f LIBC:crt0dat.obj
0001:0000078b _parse_cmdline 0040178b f LIBC:stdargv.obj

The first thing to note is that all addresses in the map file are relative to the preferred load address, 0x00400000. Therefore, we’re looking for the function which contains the address 0x00400000 + 0x00001013 = 0x00401013. To do so, we should look for the function which has the largest address less than or equal to the address of the crash. The function _DoCrash() is at the address 0x00401010, so the crash appears to be at the address _DoCrash+0x3.

Better still is to create a PDB file alongside your retail builds, as in:

1
2


C:\Proj\Crashes> cl /Zi /Od /nologo /c DerefNullPtr.c
C:\Proj\Crashes> link /DEBUG /SUBSYSTEM:CONSOLE /PDB:DerefNullPtr.pdb /MAP:DerefNullPtr.map /OUT:DerefNullPtr.exe DerefNullPtr.obj

Note that the /DEBUG option may be a little confusing — it really should named /CREATEPDB.

Unfortunately, this change will invalidate the previous crash address. (This wouldn’t be a problem if building retail PDBs was part of your normal process) However, we can reproduce the bug within the debugger:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33


C:\Proj\Crashes> windbg DerefNullPtr.exe

Microsoft (R) Windows Debugger Version 6.6.0003.5
Copyright (c) Microsoft Corporation. All rights reserved.

CommandLine: DerefNullPtr.exe
Symbol search path is: SRV\*C:\websymbols\*http://msdl.microsoft.com/download/symbols
Executable search path is:
ModLoad: 00400000 0040d000 DerefNullPtr.exe
ModLoad: 7c900000 7c9b0000 ntdll.dll
ModLoad: 7c800000 7c8f4000 C:\WINDOWS\system32\kernel32.dll
(3e4.7f4): Break instruction exception -- code 80000003 (first chance)
eax=00241eb4 ebx=7ffd9000 ecx=00000000 edx=00000001 esi=00241f48 edi=00241eb4
eip=7c901230 esp=0012fb20 ebp=0012fc94 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202
ntdll!DbgBreakPoint:
7c901230 cc int 3
0:000> g
(3e4.7f4): Access violation -- code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=003214a8 ebx=7ffd9000 ecx=00000001 edx=7c90eb94 esi=00000a28 edi=00000000
eip=00401023 esp=0012fedc ebp=0012fedc iopl=0 nv up ei pl zr na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010246
\*** WARNING: Unable to verify checksum for DerefNullPtr.exe
DerefNullPtr!DoCrash+0×3:
00401023 c6050000000001 mov byte ptr [00000000],0×1 ds:0023:00000000=??
0:000> kb
ChildEBP RetAddr Args to Child
0012fedc 00401018 0012ffc0 00401245 00000001 DerefNullPtr!DoCrash+0×3 [c:\proj\crashes\derefnullptr.c @ 3]
0012fee4 00401245 00000001 00321470 003214a8 DerefNullPtr!main+0×8 [c:\proj\crashes\derefnullptr.c @ 9]
0012ffc0 7c816fd7 00090000 00a7fa9c 7ffd9000 DerefNullPtr!mainCRTStartup+0×173 [f:\vs70builds\3077\vc\crtbld\crt\src\crt0.c @ 259]
0012fff0 00000000 004010d2 00000000 78746341 kernel32!BaseProcessStart+0×23

Using PDBs, you have the ability to map instruction addresses to a file and line number, even with retail builds. I recommend producing PDBs as part of your build process and placing them somewhere accessible by all developers on your team.

Windbg is an extremely powerful debugger included with Microsoft’s free Debugging Tools for Windows.