Document the new extra_modules feature

README.md

@@ -57,14 +57,18 @@ aggregate.csv  Multisets.csv
 Process a module (check stability of all of its methods) in a Julia session:
 
 ``` julia
-checkModule(MyModule)
+checkModule(MyModule; extra_modules = Module[])
 ```
 
 - Assumes: `MyModule` is loaded.
 - Effects: in the current directory creates:
   - `MyModule.csv` with raw results of analysis;
   - `MyModule-agg.txt` with aggregate stats (possibly, for further analysis).
 
+The `extra_modules` parameter can be used to extend the search to detect external method definitions. By default, it is an empty array, in which case we look for functions just in `MyModule`. If non-empty, we also process functions from each of the extra modules and filter their methods to only those defined in `MyModule`. This approach catches cases when methods are added to external functions, e.g. `Base.hash(x::MyType, h::Uint) = ...`.
+
+To discover all methods from a module, we need to check the module itself and also all of its transitive dependencies. However, it's not trivial to get this list of modules, so a good approximation is to pass `Base.loaded_modules_array()` as the extra modules argument. This will return all the modules loaded currently in the julia session (including `Base`, `Core`, `MyModule`, and all of `MyModule`'s transitive dependencies); any other modules that also happen to be loaded are included as well, but this doesn't seem to affect performace much.
+
 A (possibly empty) set of `agg`-files can be turned into a single CSV via calling
 `scripts/aggregate.sh`. For just one file, it only adds the heading. The result
 is written in `aggregate.csv`.

src/report.jl

@@ -121,8 +121,15 @@ aggregateStats(mcs::StCheckResults) :: AgStats = AgStats(
 # ----------------------------------------
 
 # checkModule :: Module, Path; String, Vector{Module} -> IO ()
-# Check stability in the given module (and also look for functions
-# in the extra_modules), store results under the given path
+# Check stability in the given module, store results under the given path
+#
+# The `extra_modules` parameter can be used to include functions that are bound in these modules and is needed because a module can add methods to external functions without also
+# importing them into its scope. For example, when a module `m` contains `Base.push!(x::MyVec, e) = ...` without importing `push!`, then `names(m)` does not include `push!`.
+# However, `names(Base)` does, and we can then filter only the methods that originate in `m`.
+#
+# To make sure we find all methods from `m`, `extra_modules` has to contain all the transitive dependencies of `m`, as well as `Base` and `Core` (which are always imported by default).
+# One simple over-approximation is to use `Base.loaded_modules_array()`.
+#
 # Effects:
 #   1. Module.csv with detailed, user-friendly results
 #   2. Module-agg.txt with aggregate results