Warning (Beta): The RVA feature set is in rollout and subject to change and limitations.
XEvent and other outputs (errorlog, sqldump*.txt, ...) contain call stack outputs. Some of these outputs are easy to symbolize while others are more difficult.
For example (error log output):
00007FFFF1A66B54 Module(ntdll+00000000000A6B54)
00007FFFEF609F59 Module(KERNELBASE+0000000000139F59)
00007FFF93981D96 Module(sqldk+0000000000011D96)
00007FFF93981CA6 Module(sqldk+0000000000011CA6)
00007FFF93971F9A Module(sqldk+0000000000001F9A)
00007FFF93EA1200 Module(sqlmin+0000000000001200)
00007FFF93EA321E Module(sqlmin+000000000000321E)
00007FFF9397A6C3 Module(sqldk+000000000000A6C3)
00007FFF9397A34D Module(sqldk+000000000000A34D)
00007FFF93979F3E Module(sqldk+0000000000009F3E)
00007FFF93992FA2 Module(sqldk+0000000000022FA2)
00007FFF9399318C Module(sqldk+000000000002318C)
00007FFF93993B83 Module(sqldk+0000000000023B83)
00007FFFEFDD54E0 Module(KERNEL32+00000000000154E0)
00007FFFF19C485B Module(ntdll+000000000000485B)
Using the Windows Debugger you can load the target binary and symbolize using our public symbols.
windbg.exe -z sqldk.dll
> ln sqldk+0000000000023B83
Hint - Use the SqlCallStackResolver utility (Arvind has made this easy): GitHub - microsoft/SQLCallStackResolver
00007FFFF1A66B54 Module(ntdll+00000000000A6B54)
00007FFFEF609F59 Module(KERNELBASE+0000000000139F59)
sqldk!SOS_Scheduler::SwitchToThreadWorker
sqldk!SOS_Scheduler::Switch
sqldk!SOS_Scheduler::SuspendNonPreemptive
sqlmin!SOS_Task::Sleep
sqlmin!XStoreIoRetryTask2
sqldk!SOS_Task::Param::Execute
sqldk!SOS_Scheduler::RunTask
sqldk!SOS_Scheduler::ProcessTasks
sqldk!SchedulerManager::WorkerEntryPoint
sqldk!SystemThreadDispatcher::ProcessWorker
sqldk!SchedulerManager::ThreadEntryPoint
00007FFFEFDD54E0 Module(KERNEL32+00000000000154E0)
00007FFFF19C485B Module(ntdll+000000000000485B)
Some outputs, such as the XEvent call stack action output the raw stack frame information and require a rebase to loaded module information in order to symbolize. The security feature for random address virtualization loads images at different addresses each time the image is loaded. This requires the module base address and the raw address in order to calculate the relative virtual address for symbolization.
Formula: Raw Address - Base Address = RVA address
Sample XEvent Call Stack
00007FFFF1A66B54 00007FFFEF609F59 00007FFF93981D96 00007FFF93981CA6 ...
Tips
- XEvent Callstack Action - great when doing histogram bucketing but requires dm_os_loaded_modules or the module output from a stack capture in order to symbolize. The SqlCallStackResolver can do this work as well.
- XEvent Callstack_Rva Action - Requires no additional capture to symbolize but is not as performant as the callstack action. Use for targeted debugging efforts and low production rate events (Latch timeout, deadlock, etc.)
RVA Call Stack RVA
Several of us have been adding callstack_rva capabilities, upgrading outputs, etc. For example, a latch timeout message includes the RVA callstack of the owner when available, the XEvent Callstack_RVA action is exposed, and so forth.
- Currently we are testing use of the changes with telemetry workloads and are beginning to roll the changes out (coming to a SQL Server near you, including box releases) and is restricted to specific scenarios at the current time.
- Arvind has/is upgrading the SqlCallStackResolver to accommodate the RVA format.
- dbcc stackdump (...) is receiving the SKIP_DUMP with option.
- etc.
Example Callstack_rva Output
ParentObject Object Field VALUE
Thread StackDump: RVAStack rvastack <rvastack ThreadId="33900">
Thread StackDump: RVAStack Frame <frame id="00" address="0x7FFDCD0842A4" pdb="ntdll.pdb" age="1" guid="D522F52A-E48D-7166-E35F-4FB492B6398B" module="ntdll.dll" rva="0xA42A4" />
Thread StackDump: RVAStack Frame <frame id="01" address="0x7FFDCA79FB10" pdb="kernelbase.pdb" age="1" guid="AF202873-637A-4CAA-BB4A-CB056CE0BCCA" module="KERNELBASE.dll" rva="0x4FB10" />
Thread StackDump: RVAStack Frame <frame id="02" address="0x7FFDCA79FA0E" pdb="kernelbase.pdb" age="1" guid="AF202873-637A-4CAA-BB4A-CB056CE0BCCA" module="KERNELBASE.dll" rva="0x4FA0E" />
Thread StackDump: RVAStack Frame <frame id="03" address="0x7FFCF0706C6D" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x4C6C6D" />
Thread StackDump: RVAStack Frame <frame id="04" address="0x7FFCF06BA067" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x47A067" />
Thread StackDump: RVAStack Frame <frame id="05" address="0x7FFCF06BD881" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x47D881" />
Thread StackDump: RVAStack Frame <frame id="06" address="0x7FFCF06BA4BA" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x47A4BA" />
Thread StackDump: RVAStack Frame <frame id="07" address="0x7FFCF181F6BE" pdb="sqllang.pdb" age="13" guid="6A655131-DA07-4209-A18A-F9E57A9E4AE7" module="sqllang.dll" rva="0x99F6BE" />
Thread StackDump: RVAStack Frame <frame id="08" address="0x7FFCF06BA381" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x47A381" />
Thread StackDump: RVAStack Frame <frame id="09" address="0x7FFCF06BDBFA" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x47DBFA" />
Thread StackDump: RVAStack Frame <frame id="10" address="0x7FFCF03C288F" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x18288F" />
Thread StackDump: RVAStack Frame <frame id="11" address="0x7FFCF03CE097" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x18E097" />
Thread StackDump: RVAStack Frame <frame id="12" address="0x7FFCF03CAE16" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x18AE16" />
Thread StackDump: RVAStack Frame <frame id="13" address="0x7FFCF03B142D" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x17142D" />
Thread StackDump: RVAStack Frame <frame id="14" address="0x7FFCF03ABAD3" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x16BAD3" />
Thread StackDump: RVAStack Frame <frame id="15" address="0x7FFCF03A7D69" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x167D69" />
Thread StackDump: RVAStack Frame <frame id="16" address="0x7FFCF03AF3F5" pdb="SqlDK.pdb" age="14" guid="552E11EA-6498-41E8-90EC-418C5E66469A" module="sqldk.dll" rva="0x16F3F5" />
Thread StackDump: RVAStack Frame <frame id="17" address="0x7FFDCB3F54E0" pdb="kernel32.pdb" age="1" guid="DC094362-EEB8-DA89-986E-90A2096ACE28" module="KERNEL32.DLL" rva="0x154E0" />
Thread StackDump: RVAStack Frame <frame id="18" address="0x7FFDCCFE485B" pdb="ntdll.pdb" age="1" guid="D522F52A-E48D-7166-E35F-4FB492B6398B" module="ntdll.dll" rva="0x485B" />
Thread StackDump: RVAStack </rvastack>
SKIP_DUMP
SKIP_DUMP does all the other work that dbcc stackdump normally does to generate a txt file and filter the results based on the (...) information, skipping actual dump generation. Instead, the RVA stack are generated to the output locations as well as a result set to the client. Each frame contains enough information to identify the proper pdb and resolve the stack frame.
When available you can use dbcc stackdump (...) with SKIP_DUMP to capture call stack(s) without requiring a dump.
Posted at https://sl.advdat.com/35OUuMphttps://sl.advdat.com/35OUuMp