maintain separate Ghidra state for each Python shared interpreter (#69)

* maintain seperate Ghidra state for each Python shared interpreter * more robust io redirection * additional cleanup of DB transactions * fix DB transaction * update unit tests to use GhidraScript variable functions * add additional unit tests; fix #71 * update README to include breaking changes
mandiant · Aug 18, 2023 · fdb9d0c · fdb9d0c
1 parent 4797e02
commit fdb9d0c
Show file tree

Hide file tree

Showing 14 changed files with 427 additions and 237 deletions.
diff --git a/README.md b/README.md
@@ -8,7 +8,7 @@ Check out:
 
 - The overview in our first [Ghidrathon blog post](https://www.mandiant.com/resources/blog/ghidrathon-snaking-ghidra-python-3-scripting)
 
-Ghidrathon replaces the existing Python 2 extension implemented via Jython. This includes the interactive interpreter window, integration with the Ghidra Script Manager, and script execution in Ghidra headless mode. 
+Ghidrathon replaces the existing Python 2.7 extension implemented via Jython. This includes the interactive interpreter window, integration with the Ghidra Script Manager, and script execution in Ghidra headless mode.
 
 ## Python 3 Interpreter Window
 
@@ -28,7 +28,7 @@ Ghidrathon helps you execute Python 3 scripts in Ghidra headless mode. Execute t
 
 ```
 $ analyzeHeadless C:\Users\wampus example -process example.o -postScript ghidrathon_example.py
-...
+[...]
 INFO  SCRIPT: C:\Users\wampus\.ghidra\.ghidra_10.0.3_PUBLIC\Extensions\Ghidrathon-master\ghidra_scripts\ghidrathon_example.py (HeadlessAnalyzer)
 Function _init @ 0x101000: 3 blocks, 8 instructions
 Function FUN_00101020 @ 0x101020: 1 blocks, 2 instructions
@@ -49,7 +49,7 @@ Function __libc_start_main @ 0x105010: 0 blocks, 0 instructions
 Function __gmon_start__ @ 0x105018: 0 blocks, 0 instructions
 Function _ITM_registerTMCloneTable @ 0x105020: 0 blocks, 0 instructions
 Function __cxa_finalize @ 0x105028: 0 blocks, 0 instructions
-...
+[...]
 INFO  REPORT: Post-analysis succeeded for file: /example.o (HeadlessAnalyzer)
 INFO  REPORT: Save succeeded for processed file: /example.o (HeadlessAnalyzer)
 ```
@@ -62,6 +62,12 @@ One of our biggest motivations in developing Ghidrathon was to enable use of thi
 
 ![example](./data/ghidrathon_unicorn.png)
 
+## Writing Ghidra Python 3 Scripts
+
+Ghidrathon provides a scripting experience that closely mirrors Ghidra's Java and Jython extensions which includes making `GhidraScript` state instance variables, e.g. `currentProgram`, and `FlatProgramAPI` methods, e.g. `findBytes` 
+available at the Python `builtins` scope. This means _all_ Python modules that are imported by your code have access to these variables and methods. Ghidrathon diverges slightly from Ghidra's Java and Jython extensions by exposing `GhidraScript` 
+state variables as Python function calls versus direct accesses e.g. your Python 3 code must access `currentProgram` using the function call `currentProgram()`. This small change ensures that your Python 3 code is provided the correct `GhidraScript` state variables during execution. Please see our Ghidra Python 3 script example [here](./ghidra_scripts/ghidrathon_example.py) for a closer look at writing Python 3 scripts for Ghidra.
+
 ## How does it work?
 
 Ghidrathon links your local Python installation to Ghidra using the open-source project [Jep](https://github.com/ninia/jep). Essentially your local Python interpreter is running inside Ghidra with access to all your Python packages **and** the standard Ghidra scripting API. Ghidrathon also works with Python virtual environments helping you create, isolate, and manage packages you may only want installed for use in Ghidra. Because Ghidrathon uses your local Python installation you have control over the Python version and environment running inside Ghidra.
@@ -85,7 +91,7 @@ Tool | Version |Source |
 | Ghidra | `>= 10.2` | https://ghidra-sre.org |
 | Jep | `>= 4.1.1` | https://github.com/ninia/jep |
 | Gradle | `>= 7.3` | https://gradle.org/releases |
-| Python | `>= 3.7` | https://www.python.org/downloads |
+| Python | `>= 3.8` | https://www.python.org/downloads |
 
 Note: Ghidra >= 10.2 requires [JDK 17 64-bit](https://adoptium.net/temurin/releases/).
 

diff --git a/data/ghidrathon_interp.png b/data/ghidrathon_interp.png
diff --git a/data/ghidrathon_script.png b/data/ghidrathon_script.png
diff --git a/data/ghidrathon_unicorn.png b/data/ghidrathon_unicorn.png
diff --git a/data/python/jepbuiltins.py b/data/python/jepbuiltins.py
diff --git a/data/python/jepeval.py b/data/python/jepeval.py
@@ -52,6 +52,14 @@ def _jepeval(line):
                 # e.g. for loop; cache statement to combine and compile/eval later
                 jepeval_lines = [line]
                 return True
+            except SyntaxError as err:
+                if err.msg == "unexpected EOF while parsing":
+                    # python3.8 does not raise IndentationError, TabError so we must check for a SyntaxError
+                    # with a hard-coded message
+                    jepeval_lines = [line]
+                    return True
+                else:
+                    raise err
         else:
             # we have cached statements, user must be defining a multi-line block e.g. for loop; cache
             # statement to combine and compile/eval later

diff --git a/data/python/jepinject.py b/data/python/jepinject.py
diff --git a/data/python/jepstream.py b/data/python/jepstream.py
diff --git a/data/python/jepwelcome.py b/data/python/jepwelcome.py
@@ -32,8 +32,8 @@ def format_version():
     return "%d.%d.%d" % sys.version_info[:3]
 
 
-# Assume GhidraVersion passed from Java to Python before execution
+# Assume ghidra_version passed from Java to Python before execution
 
-print(message % (format_version(), GhidraVersion))
+print(message % (format_version(), ghidra_version))
 
-del GhidraVersion
+del ghidra_version