Merge branch 'develop'

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2017-2020 Markus Gaasedelen
+Copyright (c) 2017-2021 Markus Gaasedelen
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,13 +1,14 @@
-# Lighthouse - A Code Coverage Explorer for Reverse Engineers
+# Lighthouse - A Coverage Explorer for Reverse Engineers
+
 <p align="center">
 <img alt="Lighthouse Plugin" src="screenshots/overview.gif"/>
 </p>
 
 ## Overview
 
-Lighthouse is a powerful code coverage plugin for [IDA Pro](https://www.hex-rays.com/products/ida/) and [Binary Ninja](https://binary.ninja/). As an extension of the leading disassemblers, this plugin enables one to interactively explore code coverage data in new and innovative ways when symbols or source may not be available for a given binary.
+Lighthouse is a powerful code coverage explorer for [IDA Pro](https://www.hex-rays.com/products/ida/) and [Binary Ninja](https://binary.ninja/), providing software researchers with uniquely interactive controls to study execution maps for native applications without requiring symbols or source.
 
-This plugin is labeled only as a prototype & code resource for the community. 
+This project placed 2nd in IDA's [2017 Plug-In Contest](https://hex-rays.com/contests_details/contest2017/) and was later [nominated](https://pwnies.com/lighthouse/) in the 2021 Pwnie Awards for its contributions to the security research industry.
 
 Special thanks to [@0vercl0k](https://twitter.com/0vercl0k) for the inspiration.
 
@@ -27,11 +28,31 @@ Special thanks to [@0vercl0k](https://twitter.com/0vercl0k) for the inspiration.
 
 Lighthouse is a cross-platform (Windows, macOS, Linux) Python 2/3 plugin. It takes zero third party dependencies, making the code both portable and easy to install.
 
-1. From your disassembler's python console, run the following command to find its plugin directory:
-   - **IDA Pro**: `os.path.join(idaapi.get_user_idadir(), "plugins")`
-   - **Binary Ninja**: `binaryninja.user_plugin_path()`
+Use the instructions below for your respective disassembler.
+
+## IDA Installation
+
+1. From IDA's Python console, run the following command to find its plugin directory:
+    - `import idaapi, os; print(os.path.join(idaapi.get_user_idadir(), "plugins"))`
+2. Copy the contents of this repository's `/plugins/` folder to the listed directory.
+3. Restart your disassembler.
+
+## Binary Ninja Installation
+
+Lighthouse can be installed through the plugin manager on newer versions of Binary Ninja (>2.4.2918). The plugin will have to be installed manually on older versions.
+
+### Auto Install
+
+1. Open Binary Ninja's plugin manager by navigating the following submenus:
+    - `Edit` -> `Preferences` -> `Manage Plugins`
+2. Search for Lighthouse in the plugin manager, and click the `Enable` button in the bottom right.
+3. Restart your disassembler.
+
+### Manual Install
 
-2. Copy the contents of this repository's `/plugin/` folder to the listed directory.
+1. Open Binary Ninja's plugin folder by navigating the following submenus:
+    - `Tools` -> `Open Plugins Folder...`
+2. Copy the contents of this repository's `/plugins/` folder to the listed directory.
 3. Restart your disassembler.
 
 # Usage
@@ -76,16 +97,15 @@ If there are any other actions that you think might be useful to add to this con
 
 ## Coverage ComboBox
 
-Loaded coverage data and user constructed compositions can be selected or deleted through the coverage combobox.
+Loaded coverage and user constructed compositions can be selected or deleted through the coverage combobox.
 
 <p align="center">
 <img alt="Lighthouse Coverage ComboBox" src="screenshots/combobox.gif"/>
 </p>
 
 ## HTML Coverage Report
 
-Lighthouse can generate a rudimentary HTML coverage report of the active coverage. 
-A sample report can be seen [here](https://rawgit.com/gaasedelen/lighthouse/master/testcase/report.html).
+Lighthouse can generate rudimentary HTML coverage reports. A sample report can be seen [here](https://rawgit.com/gaasedelen/lighthouse/master/testcase/report.html).
 
 <p align="center">
 <img alt="Lighthouse HTML Report" src="screenshots/html_report.gif"/>

binjastub/README.md

@@ -0,0 +1,11 @@
+# Lighthouse - A Coverage Explorer for Reverse Engineers
+
+<p align="center">
+<img alt="Lighthouse Plugin" src="https://raw.githubusercontent.com/gaasedelen/lighthouse/master/screenshots/overview.gif"/>
+</p>
+
+## Overview
+
+Lighthouse is a powerful code coverage explorer for [IDA Pro](https://www.hex-rays.com/products/ida/) and [Binary Ninja](https://binary.ninja/), providing software researchers with uniquely interactive controls to study execution maps for native applications without requiring symbols or source.
+
+For additional usage information, please check out the full [README](https://github.com/gaasedelen/lighthouse) on GitHub.

binjastub/__init__.py

@@ -0,0 +1,26 @@
+import os 
+import sys
+
+#------------------------------------------------------------------------------
+# Binary Ninja 'Plugin Manager' Stub
+#------------------------------------------------------------------------------
+#
+#    This file is an alternative loading stub created specifically to
+#    support the ability to 'easy' install Lighthouse into Binary Ninja
+#    via its 'Plugin Manager' functionality.
+#
+#    Please disregard this code / subdirectory if performing **manual**
+#    installations of Lighthouse in IDA or Binary Ninja.
+#
+
+lh_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), "..", "plugins")
+sys.path.append(lh_path)
+
+from lighthouse.util.log import logging_started, start_logging
+from lighthouse.util.disassembler import disassembler
+
+if not logging_started():
+    logger = start_logging()
+
+logger.info("Selecting Binary Ninja loader...")
+from lighthouse.integration.binja_loader import *

binjastub/plugin.json

@@ -0,0 +1,24 @@
+{
+    "api": [
+        "python3"
+    ],
+    "author": "Markus Gaasedelen",
+    "description": "A Coverage Explorer for Reverse Engineers",
+    "license": {
+        "name": "MIT",
+        "text": "Copyright (c) 2021> Markus Gaasedelen\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
+    },
+    "longdescription": "",
+    "minimumbinaryninjaversion": 2918,
+    "name": "Lighthouse",
+    "platforms": [
+        "Darwin",
+        "Linux",
+        "Windows"
+    ],
+    "pluginmetadataversion": 2,
+    "type": [
+        "helper"
+    ],
+    "version": "0.9.2"
+}

coverage/pin/CodeCoverage.cpp

@@ -1,4 +1,3 @@
-using namespace std;
 #include <iostream>
 #include <set>
 #include <string>

coverage/pin/ImageManager.cpp

@@ -1,4 +1,3 @@
-using namespace std;
 #include "ImageManager.h"
 #include "pin.H"
 

coverage/pin/Makefile

@@ -1,7 +1,7 @@
 CONFIG_ROOT := $(PIN_ROOT)/source/tools/Config
 include $(CONFIG_ROOT)/makefile.config
 
-TOOL_CXXFLAGS += -std=c++11 -Wno-format -Wno-aligned-new
+TOOL_CXXFLAGS += -std=c++11 -Wno-format
 TOOL_ROOTS := CodeCoverage
 
 $(OBJDIR)CodeCoverage$(PINTOOL_SUFFIX): $(OBJDIR)CodeCoverage$(OBJ_SUFFIX) $(OBJDIR)ImageManager$(OBJ_SUFFIX)

coverage/pin/build-x64.bat

@@ -2,11 +2,12 @@
 cls
 
 cl ^
-    /c ^
+    /c /Fo /nologo /EHa- /EHs- /GR- /GS- /Gd /Gm- /Gy /MD /O2 /Oi- /Oy- /TP /W3 /WX- /Zc:forScope /Zc:inline /Zc:wchar_t /wd4316 /wd4530 /fp:strict ^
+    /DTARGET_IA32E /DHOST_IA32E /DTARGET_WINDOWS /DWIN32 /D__PIN__=1 /DPIN_CRT=1 /D_STLP_IMPORT_IOSTREAMS /D__LP64__ ^
+    /I"%PIN_ROOT%\extras\xed-intel64\include\xed" ^
     /I%PIN_ROOT%\source\include\pin ^
     /I%PIN_ROOT%\source\include\pin\gen ^
     /I%PIN_ROOT%\source\tools\InstLib   ^
-    /I"%PIN_ROOT%\extras\xed-intel64\include\xed" ^
     /I%PIN_ROOT%\extras\components\include ^
     /I%PIN_ROOT%\extras\stlport\include ^
     /I%PIN_ROOT%\extras ^
@@ -16,9 +17,6 @@ cl ^
     /I"%PIN_ROOT%\extras\crt\include\arch-x86_64" ^
     /I%PIN_ROOT%\extras\crt\include\kernel\uapi ^
     /I"%PIN_ROOT%\extras\crt\include\kernel\uapi\asm-x86" ^
-    /nologo /W3 /WX- /O2 ^
-    /D TARGET_IA32E /D HOST_IA32E /D TARGET_WINDOWS /D WIN32 /D __PIN__=1 /D PIN_CRT=1 /D __LP64__ ^
-    /Gm- /MT /GS- /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR- /Gd /TP /wd4530 /GR- /GS- /EHs- /EHa- /FP:strict /Oi- ^
     /FIinclude/msvc_compat.h CodeCoverage.cpp ImageManager.cpp ImageManager.h TraceFile.h
 
 link ^
@@ -29,7 +27,7 @@ link ^
     /LIBPATH:%PIN_ROOT%\intel64\lib ^
     /LIBPATH:"%PIN_ROOT%\intel64\lib-ext" ^
     /LIBPATH:"%PIN_ROOT%\extras\xed-intel64\lib" ^
-    /LIBPATH:%PIN_ROOT%\intel64\runtime\pincrt pin.lib xed.lib pinvm.lib kernel32.lib "stlport-static.lib" "m-static.lib" "c-static.lib" "os-apis.lib" "ntdll-64.lib" crtbeginS.obj ^
+    /LIBPATH:%PIN_ROOT%\intel64\runtime\pincrt pin.lib xed.lib pinvm.lib pincrt.lib ntdll-64.lib kernel32.lib crtbeginS.obj ^
     /NODEFAULTLIB ^
     /MANIFEST:NO ^
     /OPT:NOREF ^

coverage/pin/build-x86.bat

@@ -2,8 +2,8 @@
 cls
 
 cl ^
-    /c /EHa- /EHs- /GR- /GS- /Gd /Gm- /Gy /MT /O2 /Oi- /Oy- /TP /W3 /WX- /Zc:forScope /Zc:inline /Zc:wchar_t /fp:precise /nologo /wd4316  ^
-    /DTARGET_IA32 /DHOST_IA32 /DTARGET_WINDOWS /DBIGARRAY_MULTIPLIER=1 /DWIN32 /D__PIN__=1 /DPIN_CRT=1 /D__i386__ ^
+    /c /Fo /nologo /EHa- /EHs- /GR- /GS- /Gd /Gm- /Gy /MD /O2 /Oi- /Oy- /TP /W3 /WX- /Zc:forScope /Zc:inline /Zc:wchar_t /wd4316 /wd4530 /fp:precise ^
+    /DTARGET_IA32 /DHOST_IA32 /DTARGET_WINDOWS /DWIN32 /D__PIN__=1 /DPIN_CRT=1 /D_STLP_IMPORT_IOSTREAMS /D__i386__ ^
     /I"%PIN_ROOT%\extras\xed-ia32\include\xed" ^
     /I%PIN_ROOT%\source\include\pin ^
     /I%PIN_ROOT%\source\include\pin\gen ^
@@ -28,7 +28,7 @@ link ^
     /LIBPATH:%PIN_ROOT%\ia32\lib ^
     /LIBPATH:"%PIN_ROOT%\ia32\lib-ext" ^
     /LIBPATH:"%PIN_ROOT%\extras\xed-ia32\lib" ^
-    /LIBPATH:%PIN_ROOT%\ia32\runtime\pincrt pin.lib xed.lib pinvm.lib kernel32.lib "stlport-static.lib" "m-static.lib" "c-static.lib" "os-apis.lib" "ntdll-32.lib" crtbeginS.obj ^
+    /LIBPATH:%PIN_ROOT%\ia32\runtime\pincrt pin.lib xed.lib pinvm.lib pincrt.lib ntdll-32.lib kernel32.lib crtbeginS.obj ^
     /NODEFAULTLIB ^
     /MANIFEST:NO ^
     /OPT:NOREF ^

plugins/lighthouse/context.py

@@ -66,6 +66,8 @@ def terminate(self):
         """
         Spin down any session subsystems before the session is deleted.
         """
+        if not self._started:
+            return
         self.painter.terminate()
         self.director.terminate()
         self.metadata.terminate()

plugins/lighthouse/director.py

@@ -666,7 +666,7 @@ def _optimize_coverage_data(self, coverage_addresses):
         #
 
         block_ratio = len(basic_blocks) / float(len(instructions))
-        block_trace_confidence = 0.90
+        block_trace_confidence = 0.80
         logger.debug("Block confidence %f" % block_ratio)
 
         #

plugins/lighthouse/integration/binja_integration.py

@@ -25,6 +25,8 @@ def __init__(self):
     def get_context(self, dctx, startup=True):
         """
         Get the LighthouseContext object for a given database context.
+
+        In Binary Ninja, a dctx is a BinaryView (BV).
         """
         dctx_id = ctypes.addressof(dctx.handle.contents)
 
@@ -65,6 +67,30 @@ def get_context(self, dctx, startup=True):
         # return the lighthouse context object for this database ctx / bv
         return lctx
 
+    def binja_close_context(self, dctx):
+        """
+        Attempt to close / spin-down the LighthouseContext for the given dctx.
+
+        In Binary Ninja, a dctx is a BinaryView (BV).
+        """
+        dctx_id = ctypes.addressof(dctx.handle.contents)
+
+        # fetch the LighthouseContext for the closing BNDB
+        try:
+            lctx = self.lighthouse_contexts.pop(dctx_id)
+
+        #
+        # if lighthouse was not actually used for this BNDB / session, then
+        # the lookup will fail as there is nothing to spindown
+        #
+
+        except KeyError:
+            return
+
+        # spin down the closing context (stop threads, cleanup qt state, etc)
+        logger.info("Closing a LighthouseContext...")
+        lctx.terminate()
+
     #--------------------------------------------------------------------------
     # UI Integration (Internal)
     #--------------------------------------------------------------------------

plugins/lighthouse/integration/binja_loader.py

@@ -30,4 +30,3 @@
 except Exception as e:
     lmsg("Failed to initialize Lighthouse")
     logger.exception("Exception details:")
-

plugins/lighthouse/integration/core.py

@@ -26,9 +26,9 @@ class LighthouseCore(object):
     # Plugin Metadata
     #--------------------------------------------------------------------------
 
-    PLUGIN_VERSION = "0.9.1"
+    PLUGIN_VERSION = "0.9.2"
     AUTHORS        = "Markus Gaasedelen"
-    DATE           = "2020"
+    DATE           = "2021"
 
     #--------------------------------------------------------------------------
     # Initialization
@@ -87,14 +87,10 @@ def print_banner(self):
 
         # build the main banner title
         banner_params = (self.PLUGIN_VERSION, self.AUTHORS, self.DATE)
-        banner_title  = "Lighthouse v%s - (c) %s - %s" % banner_params
+        banner_title  = "v%s - (c) %s - %s" % banner_params
 
         # print plugin banner
-        lmsg("")
-        lmsg("-"*75)
-        lmsg("---[ %s" % banner_title)
-        lmsg("-"*75)
-        lmsg("")
+        lmsg("Loaded %s" % banner_title)
 
     #--------------------------------------------------------------------------
     # Disassembler / Database Context Selector

plugins/lighthouse/integration/ida_loader.py

@@ -88,7 +88,7 @@ def term(self):
         except Exception as e:
             logger.exception("Failed to cleanly unload Lighthouse from IDA.")
         end = time.time()
-        print("-"*50)
+        logger.debug("-"*50)
 
         logger.debug("IDA term done... (%.3f seconds...)" % (end-start))