Debug bundles (part 1) (#370)

Changelog.md

@@ -1,20 +1,24 @@
 # Spectrometer Changelog
 
+## v2.17.0
+
+- When running `fossa analyze` with the `--debug` flag, we now create a `fossa.debug.json.gz` file containing detailed runtime traces for project discovery and dependency analysis
+
 ## v2.16.6
-- Monorepo: Add automatic retries to failed API calls. ([#392](https://github.com/fossas/spectrometer/pull/392))
+- Monorepo: Adds automatic retries to failed API calls. ([#392](https://github.com/fossas/spectrometer/pull/392))
 
 ## v2.16.5
-- JSON Output for `fossa test --json` when there are no issues. ([#387](https://github.com/fossas/spectrometer/pull/387))
+- Adds JSON Output for `fossa test --json` when there are no issues. ([#387](https://github.com/fossas/spectrometer/pull/387))
 
 ## v2.16.4
-- Monorepo: Fix bug with symlink logic mismatch between walker and buildspec uploader. ([#388](https://github.com/fossas/spectrometer/pull/388))
+- Monorepo: Fixes bug with symlink logic mismatch between walker and buildspec uploader. ([#388](https://github.com/fossas/spectrometer/pull/388))
 
 ## v2.16.3
-- Monorepo: Fix bug with non-glob exclusions. ([#386](https://github.com/fossas/spectrometer/pull/386))
+- Monorepo: Fixes bug with non-glob exclusions. ([#386](https://github.com/fossas/spectrometer/pull/386))
 
 ## v2.16.2
-- Monorepo: Don't crash if there are no ninja/buildspec files to upload. ([#385](https://github.com/fossas/spectrometer/pull/385))
-- Monorepo: Fix issue with only-path/exclude-path globs.
+- Monorepo: Fixes crash when there are no ninja/buildspec files to upload. ([#385](https://github.com/fossas/spectrometer/pull/385))
+- Monorepo: Fixes issue with only-path/exclude-path globs.
 
 ## v2.16.1
 - Gradle: Supports analysis of projects using gralde v3.3 or below. ([#370](https://github.com/fossas/spectrometer/pull/370))

devdocs/replay-logging.md

@@ -8,16 +8,16 @@ It leverages some unique properties of haskell that allow us to "record" an end-
 
 ## Recording
 
-The `--record` flag is used to run `fossa analyze` in record mode. The analysis will run normally (if a bit slower), and create a file called `fossa.debug.json`.
+The `--debug` flag is used to run `fossa analyze` in record mode. The analysis will run normally (if a bit slower), and create a file called `fossa.debug.json`.
 
-In addition to some basic metadata, `fossa.debug.json` contains the arguments and results of each method invocation from our `ReadFS` and `Exec` effects:
+In addition to some basic metadata, `fossa.debug.json` contains the arguments and results of each method invocation from our `ReadFS` and `Exec` effects within the `journals` key:
 
 ```json
 {
-  "commit": "0abcdef",
+  "system": {...},
   "workdir": "/foo",
   "args": ["analyze", "--record"],
-  "effects": {
+  "journals": {
     "ReadFS": [...]
     "Exec": [...]
   }
@@ -26,6 +26,8 @@ In addition to some basic metadata, `fossa.debug.json` contains the arguments an
 
 ## Replaying
 
+**Replay mode is temporarily unavailable.**
+
 The `--replay <file>` flag is used to run `fossa analyze` in replay mode. Rather than reading from real files with our `ReadFS` effect or running real commands with our `Exec` effect, we stub in the recorded effect calls from `fossa.debug.json`.
 
 For replay mode to work, we need to "run analysis" in the same directory (or a subdirectory) of the one used when recording `fossa.debug.json`. In the example above, this directory is `/foo`: `fossa analyze --replay fossa.debug.json /foo`. It's recommended to also use the same CLI commit and arguments (minus `--record`, of course).

spectrometer.cabal

@@ -33,7 +33,6 @@ common lang
     GADTSyntax
     GeneralizedNewtypeDeriving
     HexFloatLiterals
-    ImportQualifiedPost
     InstanceSigs
     KindSignatures
     LambdaCase
@@ -47,12 +46,13 @@ common lang
     RankNTypes
     ScopedTypeVariables
     StandaloneDeriving
-    StandaloneKindSignatures
     StrictData
     TupleSections
     TypeApplications
     TypeOperators
     TypeSynonymInstances
+    ImportQualifiedPost
+    StandaloneKindSignatures
 
   ghc-options:
     -Wall -Wincomplete-uni-patterns -Wcompat
@@ -131,14 +131,16 @@ library
   exposed-modules:
     Algebra.Graph.AdjacencyMap.Extra
     App.Docs
+    App.Fossa.API.BuildLink
+    App.Fossa.API.BuildWait
     App.Fossa.Analyze
+    App.Fossa.Analyze.Debug
     App.Fossa.Analyze.Graph
     App.Fossa.Analyze.GraphBuilder
     App.Fossa.Analyze.GraphMangler
     App.Fossa.Analyze.Project
     App.Fossa.Analyze.Record
-    App.Fossa.API.BuildLink
-    App.Fossa.API.BuildWait
+    App.Fossa.Analyze.Types
     App.Fossa.ArchiveUploader
     App.Fossa.BinaryDeps
     App.Fossa.BinaryDeps.Jar
@@ -176,13 +178,15 @@ library
     App.OptionExtensions
     App.Pathfinder.Main
     App.Pathfinder.Scan
+    App.Pathfinder.Types
     App.Types
     App.Util
     App.Version
     App.Version.TH
     Console.Sticky
     Control.Carrier.AtomicCounter
     Control.Carrier.AtomicState
+    Control.Carrier.Debug
     Control.Carrier.Diagnostics
     Control.Carrier.Diagnostics.StickyContext
     Control.Carrier.Finally
@@ -194,6 +198,7 @@ library
     Control.Effect.AtomicCounter
     Control.Effect.AtomicState
     Control.Effect.ConsoleRegion
+    Control.Effect.Debug
     Control.Effect.Diagnostics
     Control.Effect.Finally
     Control.Effect.Output
@@ -225,9 +230,9 @@ library
     Effect.ReadFS
     Fossa.API.Types
     Graphing
-    Path.Extra
-    Parse.XML
     Network.HTTP.Req.Extra
+    Parse.XML
+    Path.Extra
     Srclib.Converter
     Srclib.Types
     Strategy.Android.Util
@@ -291,8 +296,8 @@ library
     Strategy.Python.SetupPy
     Strategy.Python.Setuptools
     Strategy.Python.Util
-    Strategy.Rebar3
     Strategy.RPM
+    Strategy.Rebar3
     Strategy.Ruby.BundleShow
     Strategy.Ruby.GemfileLock
     Strategy.Scala
@@ -353,9 +358,9 @@ test-suite unit-tests
     Conda.EnvironmentYmlSpec
     Control.Carrier.DiagnosticsSpec
     Dart.PubDepsSpec
-    Dart.PubSpecSpec
     Dart.PubSpecLockSpec
     Discovery.ArchiveSpec
+    Dart.PubSpecSpec
     Discovery.FiltersSpec
     Effect.ExecSpec
     Elixir.MixTreeSpec
@@ -371,8 +376,8 @@ test-suite unit-tests
     Go.TransitiveSpec
     Googlesource.RepoManifestSpec
     Gradle.GradleSpec
-    GraphingSpec
     GraphUtil
+    GraphingSpec
     Haskell.CabalSpec
     Haskell.StackSpec
     Maven.DepTreeSpec