Friday, April 3, 2009

Tamarin-Tracing and fun with Windows API’s

Between all the fun of traveling and chugging away at my Master's thesis I managed to dig up my old Tamarin-Tracing code. When I tried to run it, to my great surprise I got an assertion error. The surprise came from the fact that code used to work, and except for an OS environment is the same. Before I was using XP Pro, and now I am using Vista Business. After the initial reaction of "Forget about I am going for a bike ride." I started to dig in to the code. The assertion itself happened in GCMemoryProfiler::ChangeSizeExplicit function.

GCAssert(!g_traceTable[traceIndex].type_name || strcmp(g_traceTable[traceIndex].type_name, nm) == 0);

This function is executed only when Debug mode is enabled. Since this is how I roll, it needed to be fixed. It turns out the underlying cause of this problem was a call to the Windows API StackWalk64 in the GCHeapWin::GetStackTrace function. As it turns out in Vista it exited too early, which caused trace to be set to 0. At which point some later code assumed that trace mode was disabled and an index of 0 was returned. For ChangeSizeExplicit this is not a correct behavior since either trace at index needs to be null or type_name needs to match to what is passed in.

At this point once again Google and MSDN came to the rescue. Apparently the "optional", as in I guess it was optional before, parameter AddrStack in STACKFRAME64 data structure needs to be initialized. The changes are very minimal. This code:

__asm

{

call x

x: pop eax

mov c.Eip, eax

mov c.Ebp, ebp

}


frame.AddrPC.Offset = c.Eip;

frame.AddrPC.Mode = AddrModeFlat;

frame.AddrFrame.Offset = c.Ebp;

frame.AddrFrame.Mode = AddrModeFlat;

turns in to:

__asm

{

call x

x: pop eax

mov c.Eip, eax

mov c.Ebp, ebp

mov c.Esp, esp

}

frame.AddrPC.Offset = c.Eip;

frame.AddrPC.Mode = AddrModeFlat;

frame.AddrFrame.Offset = c.Ebp;

frame.AddrFrame.Mode = AddrModeFlat;

frame.AddrStack.Offset = c.Esp;

frame.AddrStack.Mode = AddrModeFlat;

After the change everything worked once again.