All software has bugs. When a bug in an application manifests itself, it affects just that one application. Whereas, a bug in a kernel mode driver affects the entire system and often leads to the infamous Blue Screen of Death (BSOD). The color of that infamous screen may have changed in recent versions of Windows, but the underlying causes of failure have not.
This course is targeted at security researchers, software developers, support engineers who must regularly analyze and debug Windows kernel mode software. This course builds the necessary foundation for effective kernel debugging with topics such as configuring the debugger, debug symbols, performing basic debugging tasks, debugger scripting, retrieving function parameters from x64 stack, mapping x64 assembler to high-level language constructs, etc. It then dives into various techniques and strategies that can be applied to perform triaging, fault isolation, root cause analysis of crashes and hangs caused by kernel mode drivers. Also, this course touches upon the identification of malicious behavior in the kernel commonly exhibited by rootkits.
In the hands-on lab exercises, students work on a wide variety of kernel mode crash and hang dumps that have been captured on various versions of Windows ranging from Windows XP to Windows 10. Each memory dump involves applying Windows internals knowledge and a unique set of debugging techniques to go from "!analyze-v" to isolating the module responsible for the crash or hang to determining the root cause of the problem and potential ways of fixing it.
Students debug memory dumps that cover a wide range of bug-check codes such as BAD_POOL_CALLER, IRQL_NOT_LESS_OR_EQUAL, SYSTEM_THREAD_EXCEPTION_NOT_HANDLED, UNEXPECTED_KERNEL_MODE_TRAP, KERNEL_MODE_EXCEPTION_NOT_HANDLED, NO_MORE_IRP_STACK_LOCATIONS, KERNEL_APC_PENDING_DURING_EXIT, ATTEMPTED_SWITCH_FROM_DPC, DRIVER_POWER_STATE_FAILURE, CRITICAL_STRUCTURE_CORRUPTION, ATTEMPTED_EXECUTE_OF_NOEXECUTE_MEMORY and last but not the least MANUALLY_INITIATED_CRASH for system hang scenarios.
To maximize student engagement, the course delivery is a mix of theory, instructor-led demos, lab exercises involving debugging and analysis of memory dumps, and quizzes to check students’ grasp of the content.
This training course focuses on software and security related drivers and does NOT cover issues related to hardware drivers for PCI, USB, Bluetooth devices.
Attendees must have a good working knowledge of the windows kernel.
The CodeMachine Windows Kernel Internals course provides the prerequisite Windows kernel knowledge required to get the maximum value from this course.
The objective of this section is to learn about the kernel debugger, debugging symbols, and debugger usage. It covers topics such as configuring a system for kernel debugging, ramifications of kernel debugging, symbol (.PDB) file contents, resolving issues related to symbol files, using third party debugger extensions, differences between the various memory dump types (complete, kernel, active, automatic) and generating process and system memory dumps manually.
The objective of this section is to learn about the x64 CPU, registers, instructions, and the basics of reading and understanding x64 assembler required for reverse engineering. It covers topics such as x64 instruction set, x64 instruction encoding, absolute and relative offsets, little-endian vs big-endian, sign and zero extension, most frequently used instruction types (data transfer, arithmetic, logical, control flow, etc.), condition flags and conditional jumps, direct and indirect control transfers, basic blocks, control flow graphs.
The objective of this section is to learn how stacks work on 64-bit systems and how to use knowledge of 64-bit stack layout to retrieve information from the stack. It covers topics such as x64 calling convention, components of x64 stack frame, stack canaries, local variables and parameters, parameter homing space, interpreting the x64 call stack displayed by the debugger, retrieving register-based parameters from the stack and compiler flags affecting stack frame generation.
The objective of this section is to learn how to map x64 assembler to high-level language (C/C++) code constructors. It covers topics such as CPU register usage, identifying access to function parameters, local variables, global variables, structures fields, array elements, list nodes, etc., determining branch conditions, backtracking execution based on register and memory state, identifying code transformation due to compiler optimizations.
The objective of this section is to learn about the debugger's scripting support and how to use it to automate common debugging and analysis tasks. It covers topics such as appropriate usage of MASM and C++ expression evaluators, debugger command pipelines, rules for developing and executing debugger scripts, using pseudo-registers, and aliases to reduce script complexity, formatting debugger output and other automation techniques.
The objective of this section is to learn about why and how bug-checks happen, bug triaging, and fault isolation. It covers topics such as conditions leading to system bugchecks, common causes of dump generation failures, interpreting the information from debuggers’ automated analysis, using bugcheck information to determine the appropriate register context, performing sanity check on registers, obtaining system state using debugger extensions to identify runtime anomalies, triaging bugs and isolating faulty modules.
The objective of this section is to learn about debugging and analyzing systems hangs and performance issues. It covers topics such as types of hangs, waitable locks available in kernel mode (mutexes, fast mutexes, ERESOURCES, push-locks), spin-locks (exclusive spin-locks, reader writer), causes of deadlocks, deadlock detection, dependency analysis, debugging blocked power state transitions, identifying stalls in the storage path, tracking pending I/O requests, identifying overconsumption of system resources and finding the culprits.
The objective of this section is to learn deep analysis techniques that can be used to root cause problems into specific bugs in kernel modules. It covers topics such as recognizing stack patterns, determining debugging workflow, debugging invalid memory access at various IRQLs, common causes of pool corruption, identifying corruption patterns, debugging double faults, debugging multi-driver interactions, finding artifacts of kernel mode rootkits.
The objective of this section is to learn about live debugging tools that can be used to collect information that can help with debugging crashes and hangs. It covers topics such as usage of driver verifier to enable runtime verification of drivers, analyzing driver verifier bug-checks, debugging problems caused by improper resource cleanup, adding custom instrumentation to drivers, capturing live triage and full memory dumps.