Merge branch 'main' of https://github.com/mandiant/STrace

.github/workflows/build.yml

@@ -4,12 +4,12 @@ on: [push, pull_request]
 
 jobs:
   build:
-    runs-on: windows-2019
+    runs-on: windows-latest
     steps:
     - name: Checkout
       uses: actions/checkout@v2
     - name: Add msbuild to PATH
-      uses: microsoft/setup-msbuild@v1.0.2
+      uses: microsoft/setup-msbuild@v1.1
     - name: Setup rust toolchain
       uses: actions-rs/[email protected]
       with:

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 MANDIANT
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -9,7 +9,7 @@ Dtrace on windows supports multiple probe types. These include syscall, fbt, etw
 
 This reimplementation completely discards the D scripting language (note: NOT DLang the more popular modern language). The complexity of a VM in the kernel like the original DTrace implementation is unsuitable for this project. Instead, this implementation directly exposes the relevant C callbacks required to plug into the DTrace windows kernel interfaces. To enable 'hot-loading' of scripts a DLL based plugin system was used as a replacement for the VM + scripting evironment of the original dtrace. This plugin system accepts a 'normal' usermode DLL without any security checks enabled, or otherwise external dependencies and manually maps it into the kernel adress space. The `plugin dll` has exports which are invoked when kernel syscall callbacks occur. Kernel APIs are resolved via the normal import table (IAT) of the DLL, plugin DLLs link to `ntoskrnl.lib` and then the driver will resolve these APIs at load time, allowing any system apis to be called within plugin DLLs like normal. Performance of this plugin system is excellent as native code, rather than a script interpreter or a JIT, is directly executing between syscall ENTRY and RETURN. This design improves performance compared to the dtrace implementation provided by Microsoft.
 
-Setting hooks is very simple, you get a routine to register/unregister a hook by api name, with a pre and post syscall callback. Callbacks have arguments and return values accessible as read only values. Return values cannot be spoofed, and the original syscall cannot (without hacks) be 'cancelled' - this API acts as an observer. When events occur and your hook callback fires, whatever you like can be done. Callbacks are synchonous, meaning if you delay execution in the entry callback by say sitting in a while loop then the syscall call will be delayed by that amount of time. Execution of the system call occurs just after return from the entry callback, and just before entry of the return callback. This system is fully patchguard compatible, however DSE must be disable as Microsoft unfortunately considers this type of kernel extension part of the NT kernel and so validates that the root signer is windows. This _does not_ work with tricks like enabling custom kernel signers, DSE really must be off during kernel boot. 
+Setting hooks is very simple, you get a routine to register/unregister a hook by api name, with a pre and post syscall callback. Callbacks have arguments and return values accessible as read only values. Return values can be spoofed in return probes and arguments can be modified in the entry probes, but the original syscall cannot (without hacks) be 'cancelled' - this API acts as an observer. When events occur and your hook callback fires, whatever you like can be done. Callbacks are synchonous, meaning if you delay execution in the entry callback by say sitting in a while loop then the syscall call will be delayed by that amount of time. Execution of the system call occurs just after return from the entry callback, and just before entry of the return callback. This system is fully patchguard compatible, however DSE must be disable as Microsoft unfortunately considers this type of kernel extension part of the NT kernel and so validates that the root signer is windows. This _does not_ work with tricks like enabling custom kernel signers, DSE really must be off during kernel boot. 
 ```
 ValidationFlags=IMGP_LATEST_MS_ROOT_REQUIRED | IMGP_WINDOWS_ROOT_REQUIRED | IMGP_MS_SIGNATURE_REQUIRED 
 Scenario=ImgSigningScenarioWindows
@@ -48,4 +48,4 @@ Forgetting to disable DSE on boot will win you a trip to the  Automatic Repair m
 
 # Plugins
 
-To develop your own plugins it's best to use one of the existing plugins as a base project. The visual studio projects are set with many very specific settings to generate free standing binaries with no dependencies. The non-default settings are too many to list, so simply copy one of the projects, and modify the code to add your own logic instead. **If you create a useful plugin, please submit a PR**! The more plugins that are made the more useful this system is to everyone!
+To develop your own plugins it's best to use one of the existing plugins as a base project. The visual studio projects are set with many very specific settings to generate free standing binaries with no dependencies. The non-default settings are too many to list, so simply copy one of the projects, and modify the code to add your own logic instead (https://stackoverflow.com/questions/884255/visual-studio-copy-project). **If you create a useful plugin, please submit a PR** ! The more plugins that are made the more useful this system is to everyone!

blog/STrace.md

@@ -152,7 +152,7 @@ The routine ```KeSetSystemServiceCallback``` is of core interest for tracing sys
 
 ## KeSetSystemServiceCallback - Registering callbacks
 
-To register either an entry or return callback to a system call this routine accepts the name of a syscall without a prefix, a boolean to specify if it's an entry or return registration, a callback function pointer which must match one of the pointers provided to ```KiDynamicTraceCallouts```, and a metadata probe ID chosen arbitrarily by the user as a handle. The syscall name must be provided without the first two characters of the syscall, omitting the common ```Zw``` or ```Nt``` prefix. To unregister a probe the same arguments should be used as in registration except the last two arguments ```callback``` and ```probeId``` should be zero. After successfully registering a syscall callback the kernel will transition from ```KiSystemCall64``` to first ```KiTrackSystemCallEntry```. This routine will compare the service routines pointer to a tree, then if a callback is registered for that syscall, will invoke the callback registered for syscall entry in the ```KiDynamicTraceCallouts``` table. The arguments passed to this routine will be discussed later. After calling the entry callback ```KiTrackSystemCallEntry``` returns and the system service routine is executed as normal with the return value is recorded. ```KiTrackSystemCallExit``` is then executed and passed the captured return value, as well as the tree node found during the entry routine which is used to directly check if a return callback is registered without traversing the callback tree again. If a return probe is enabled on the tree node the return entry in the ```KiDynamicTraceCallouts``` is executed. Then ```KiTrackSystemCallExit``` returns and ```KiSystemCall64``` completes as normal. Because the entry and return callbacks are checked on the usermode to kernel boundary function ```KiSystemCall64``` this design can only track usermode programs as they execute system calls such as through ntdll or manually, but not API calls a driver may make. 
+To register either an entry or return callback to a system call this routine accepts the name of a syscall without a prefix, a boolean to specify if it's an entry or return registration, a callback function pointer which must match one of the pointers provided to ```KiDynamicTraceCallouts```, and a metadata probe ID chosen arbitrarily by the user as a handle. The syscall name must be provided without the first two characters of the syscall, omitting the common ```Zw``` or ```Nt``` prefix. To unregister a probe the same arguments should be used as in registration except the last two arguments ```callback``` and ```probeId``` should be zero. After successfully registering a syscall callback the kernel will transition from ```KiSystemCall64``` to first ```KiTrackSystemCallEntry```. This routine will compare the service routines pointer to a tree, then if a callback is registered for that syscall, will invoke the callback registered for syscall entry in the ```KiDynamicTraceCallouts``` table. The arguments passed to this routine will be discussed later. After calling the entry callback ```KiTrackSystemCallEntry``` returns and the system service routine is executed as normal with the return value is recorded. ```KiTrackSystemCallExit``` is then executed and passed the captured return value, as well as the tree node found during the entry routine which is used to directly check if a return callback is registered without traversing the callback tree again. If a return probe is enabled on the tree node the return entry in the ```KiDynamicTraceCallouts``` is executed. Then ```KiTrackSystemCallExit``` returns and ```KiSystemCall64``` completes as normal. Because the entry and return callbacks are checked on the usermode to kernel boundary function ```KiSystemCall64``` this design can only track usermode programs as they execute system calls such as through ntdll or manually. Zw style driver apis can be traced as well as they go through ```KiServiceInternal``` which shareds the same code path to the tracing logic inside ```KiSystemCall64```
 
 ![Invoke Callback](InvokeCallbacks.png)
 
@@ -369,4 +369,4 @@ Typically to capture a stack trace from the kernel the API `RtlVirtualUnwind` is
 
 ## DTrace's ustack()
 
-The `ustack()` command in the DTrace scripting language is implemented by the driver routine `TraceWalkUserStack`. This routine retreives the `KTRAP_FRAME` using `PsGetBaseTrapFrame` then reads each register from this structure by looping and calling `KiGetTrapFrameRegister`. The `RSP` register value is used as a starting value for the stack walk. The `InMemoryOrderModuleList` is then walked to find the module that generated the current stack frame, and the exception directory is located. `TpLookupModule` and `TpLocateExceptionDirectory` implement this module lookup, and they use the `TraceAccessMemory` api to safely access potentially invalid memory. The exception directory information of the located module is then unwound using `TpUnwindFunction`, I'm assuming this actually executes the unwind opcodes but I couldn't figure this out in detail as that machinery is poorly documented. The loop continues unwinding in this way until the desired call stack trace depth is reached, each level of the call stack recording the return address. Once the call stack addresses are all recorded, they're shipped back to usermode where a custom symbol server loads the necessary PDBs and symbolicates the stack. Since symbolication is done in user-mode this process is greatly simplified than if it had to be done in the kernel.
+The `ustack()` command in the DTrace scripting language is implemented by the driver routine `TraceWalkUserStack`. This routine retreives the `KTRAP_FRAME` using `PsGetBaseTrapFrame` then reads each register from this structure by looping and calling `KiGetTrapFrameRegister`. The `RSP` register value is used as a starting value for the stack walk. The `InMemoryOrderModuleList` is then walked to find the module that generated the current stack frame, and the exception directory is located. `TpLookupModule` and `TpLocateExceptionDirectory` implement this module lookup, and they use the `TraceAccessMemory` api to safely access potentially invalid memory. The exception directory information of the located module is then unwound using `TpUnwindFunction`, I'm assuming this actually executes the unwind opcodes but I couldn't figure this out in detail as that machinery is poorly documented. The loop continues unwinding in this way until the desired call stack trace depth is reached, each level of the call stack recording the return address. Once the call stack addresses are all recorded, the kernel driver contains a symbol cache populated by the usermode component dtrace.exe. This symbol cache is consulted to symbolicate the return addresses and then the resulting trace is eventually shipped back to usermode like all other log events.