Merge pull request #874 from mr-tz/doc/language-strings

Document language strings and fix minor bug

README.md

@@ -8,18 +8,20 @@
 
 # FLARE Obfuscated String Solver
 
+The FLARE Obfuscated String Solver (FLOSS, formerly FireEye Labs Obfuscated String Solver) uses advanced
+static analysis techniques to automatically extract and deobfuscate all strings from
+malware binaries. You can use it just like `strings.exe` to enhance the
+basic static analysis of unknown binaries.
+
+### Obfuscated Strings
+
 Rather than heavily protecting backdoors with hardcore packers, many
 malware authors evade heuristic detections by obfuscating only key
 portions of an executable. Often, these portions are strings and resources
 used to configure domains, files, and other artifacts of an infection.
 These key features will not show up as plaintext in the output of the `strings.exe` utility
 that we commonly use during basic static analysis.
 
-The FLARE Obfuscated String Solver (FLOSS, formerly FireEye Labs Obfuscated String Solver) uses advanced
-static analysis techniques to automatically deobfuscate strings from
-malware binaries. You can use it just like `strings.exe` to enhance the
-basic static analysis of unknown binaries.
-
 FLOSS extracts all the following string types:
 1. static strings: "regular" ASCII and UTF-16LE strings
 2. stack strings: strings constructed on the stack at run-time
@@ -32,32 +34,43 @@ Our [blog post](https://www.mandiant.com/resources/automatically-extracting-obfu
 
 FLOSS version 2.0 updates are detailed in this [blog post](https://www.mandiant.com/resources/floss-version-2).
 
+### Language-specific Strings
+Not all compilers use string formats that the classic `strings.exe` algorithm supports. For example, if strings are UTF-8 encoded or stored without a NULL-terminator. FLOSS can identify and extract strings from programs compiled from the following languages:
+ 1. Go
+ 2. Rust
 
-## Quick Run
-To try FLOSS right away, download a standalone executable file from the releases page:
-https://github.com/mandiant/flare-floss/releases
+The strings FLOSS extracts specific to a compiler are must easier to inspect by humans. 
 
-For a detailed description of *installing* FLOSS, review the documentation
- [here](doc/installation.md).
+Please consult the documentation to learn more about the [language-specific string extraction](doc/language_specific_strings.md).
+
+## Installation
+To use FLOSS, download a standalone executable file from the releases page:
+https://github.com/mandiant/flare-floss/releases
 
+See the [installation documentation](doc/installation.md) for a detailed description of all methods to install FLOSS.
 
-## Usage
+## Usage Examples
 Extract obfuscated strings from a malware binary:
 
-    $ floss /path/to/malware/binary
+    $ floss malware.exe
 
-Display the help/usage screen to see all available switches.
+Only extract stack and tight strings:
 
-    $ floss -h
+    $ floss --only stack tight -- suspicious.exe
 
-For a detailed description of *using* FLOSS, review the documentation
- [here](doc/usage.md).
+Do not extract static strings:
+
+    $ floss --no static -- backdoor.exe
 
-For a detailed description of *testing* FLOSS, review the documentation
- [here](doc/test.md).
+Display the help/usage screens:
 
+    $ floss -h  # show core arguments
+    $ floss -H  # show all supported arguments
+
+For a detailed description of using FLOSS, review the documentation
+ [here](doc/usage.md).
 
 ## Scripts
-FLOSS also contains additional Python scripts in the [scripts](scripts) folder 
+FLOSS also contains additional Python scripts in the [scripts](scripts) directory 
 which can be used to load its output into other tools such as Binary Ninja or IDA Pro.
 For detailed description of these scripts review the documentation [here](scripts/README.md).

doc/usage.md

@@ -7,7 +7,7 @@ You can use FLOSS just like you'd use `strings.exe`
 The enhancement that FLOSS provides is that it statically
  analyzes executable files and decodes obfuscated strings.
 These include:
-* strings encrypted in global memory, deobfuscated onto the heap
+* strings encrypted in global memory or deobfuscated onto the heap
 * strings manually created on the stack (stackstrings)
 * strings created on the stack and then further modified (tight strings)
 
@@ -37,6 +37,10 @@ containing shellcode.
 
 By default, FLOSS uses a minimum string length of four (4).
 
+### Language-specific strings
+FLOSS can identify programs compiled from selected programming languages and extract strings that are easier to inspect by humans.
+
+By default, this process is automatic. However, you can use the `--language` argument to manually select or disable this feature.
 
 ### Disable string type extraction (`--no {static,decoded,stack,tight}`)
 
@@ -70,16 +74,7 @@ Please note that `--no` and `--only` cannot be used at the same time.
 
 Write FLOSS results to `stdout` structured in JSON to make it easy to ingest by a script.
 
-    floss.exe -j malware.exe
-
-
-### Write output to a file (`-o/--output`)
-
-Write FLOSS results to a provided output file path instead of `stdout`.
-
-    floss.exe -o malware_floss_results.txt malware.exe
-    floss.exe -j -o malware_floss_results.json malware.exe
-
+    floss.exe -j malware.exe > malware_strings.json
 
 ### Load FLOSS results (`-l/--load`)
 

floss/language/utils.py

@@ -335,6 +335,9 @@ def get_struct_string_candidates_with_pointer_size(pe: pefile.PE, buf: bytes, ps
     else:
         raise ValueError("unsupported pointer size")
 
+    if not buf:
+        return
+
     limit = get_max_section_size(pe)
     low, high = get_image_range(pe)
 

floss/main.py

@@ -200,7 +200,7 @@ def make_parser(argv):
         type=str,
         choices=[l.value for l in Language if l != Language.UNKNOWN] + ["none"],
         default="",
-        help="use language-specific string extraction" if show_all_options else argparse.SUPPRESS,
+        help="use language-specific string extraction, disable using 'none'" if show_all_options else argparse.SUPPRESS,
     )
     advanced_group.add_argument(
         "-l",