Windows User Mode Internals, Debugging and Dump Analysis

5 days lecture + hands-on lab

Target Audience

Application Developers, Support Engineers and Software QA Engineers

Description

This course teaches architecture and internals of the Windows operating system with emphasis on production debugging of user mode applications and services. It helps attendees understand the behind the scenes working of the Windows operating system and debug common crashes and hangs that occur during user mode code execution.

The hands-on lab familiarizes attendees with the debugging and instrumentation tools, relevant debugger extension commands, interpretation of the command's output to investigate the state of processes, debugging techniques to isolate faulting modules and root cause crashes and hangs caused by applications and services.
NOTE: This course does NOT cover internals and debugging of .NET (managed) applications.

Pre-requisites

Attendees who must be familiar with basic operating concepts like processes, threads, virtual memory, synchronization, object and handles. To get the most value from the course attendees must be familiar with Win32 API and C programming language.

Attendees who do not have a developer (programming) background should attend the Windows Internals and Troubleshooting course.

Goals

Upon completion of this course attendees would be able to:

Configure the debugger for the “Debugging Tools for Windows” package and perform basic debugging tasks like call stacks, memory contents, navigate through data structures etc.

Understand how dumps are generated during an application crash or hang and use the appropriate tools to generate user mode memory dumps.

Understand the contents of a user mode dump file, the implication it has on debugging and configure the tools to write out the appropriate contents to the memory dumps.

Understand the information displayed by the debuggers’ automated analyzer, distinguish hangs vs. crashes, identify subsequent analysis steps and isolate the faulting modules.

Perform debugging in the absence of source code by examining the CPU assembler instructions, recognizing common instructions patterns and apply simple reverse engineering principles to debug access violation exceptions, which are the most common application crashes.

Understand the data structures the operating system uses to represent processes and thread and identify them in the debugger and interpret their contents to analyze application state.

Understand the components that make up application address space, and identify issues like address space depletion, address space fragmentation, large commits etc.

Understand the layout of thread stacks, components of a function’s stack frame, interpret the output of the debugger stack trace and retrieve function parameter and local variable information from the call stack. Use this information to manually reconstruct corrupted stacks, debug stack corruption and stack overflow problems.

Understand the causes of heap corruption, analyze corrupted heap to identify the culprit and use tools like page heap nail down heap corruption issues. Understand application’s heap usage pattern, identify symptoms of heap leaks and use tools like “user mode stack trace database” to find the source of leaks.

Understand the internals of critical sections and SRWLocks, identify deadlock scenarios by establishing cyclic dependencies and debug issues like abandoned critical sections.

Understand the relationship between objects and handles and the cost associated with creating handles. Indentify symptoms of handle leaks and use tools like handle tracing to find the offending module and function.

Understand Windows services architecture and some of the unique challenges in debugging services. Isolate shared services and debug service startup failures, service hangs and unexpected termination of services.

Understand the internals of Windows RPC, COM, DCOM, RPC protocol sequences like LPC/ALPC, Named Pipes and Sockets. Identify normal vs. stuck threads, to establish scope for debugging deadlocks. Collect RPC debugging information and traverse chains of waiting threads to isolate complex deadlocks involving multiple applications.

Topics