Skip to content

Releases: streamingfast/substreams

v1.10.5

19 Sep 14:11
Compare
Choose a tag to compare
  • Fixed an(other) issue where multiple stores running on the same stage with different initialBlocks will fail to proress (and hang)

v1.10.4

16 Sep 18:56
Compare
Choose a tag to compare

Server

  • Fix bug where some invalid cursors may be sent (with 'LIB' being above the block being sent) and add safeguard/loggin if the bug appears again
  • Fix panic in the whole tier2 process when stores go above the size limit while being read from "kvops" cached changes

CLI

  • Add -o cursor output type to substreams run for debugging purposes

v1.10.3

13 Sep 20:21
Compare
Choose a tag to compare

Server

  • Fix "cannot resolve 'old cursor' from files in passthrough mode" error on some requests with an old cursor
  • Fix handling of 'special case' substreams module with only "params" as its input: should not skip this execution (used in graph-node for head tracking)
    -> empty files in module cache with hash d3b1920483180cbcd2fd10abcabbee431146f4c8 should be deleted for consistency

CLI

  • Add substreams tools default-endpoint {network-name} to help with auto-configuration tools
  • Bump substreams init protocol version to "1" to be compatible with new codegen endpoint

v1.10.2

06 Sep 19:02
Compare
Choose a tag to compare

Tiny fix over v1.10.1:

See https://github.com/streamingfast/substreams/releases/tag/v1.10.1 for the much bigger changelog !

  • substreams gui: fix panic in some conditions when streaming from block 0

v1.10.1

06 Sep 18:00
Compare
Choose a tag to compare

Server

Note Since a bug that affected substreams with "skipping blocks" was corrected in this release, any previously produced substreams cache should be considered as possibly corrupted and be eventually replaced

  • Fix handling of modules that receive both filtered AND unfiltered data as their inputs -> some "repeated entries" could appear where no data should have showed up
  • Fix stalling on substreams with both map and store with different initialBlocks on the same stage
  • Fix: prevent execution of modules that should be skipped when running live or dev mode (different outputs than when running in batch mode on tier2)

Client

  • substreams gui fixed a panic occuring if the given package path doesn't exist
  • substreams init must now be called from within your project folder (it no longer downloads file in a subdirectory)
  • (since v1.10.0) substreams gui no longer accepts "output_module" as a single argument. It either receives nothing, the package, or the package followed by the output_module

v1.10.0

28 Aug 14:49
Compare
Choose a tag to compare

Server

  • Add sf.substreams.rpc.v2.EndpointInfo/Info endpoint (if the infoserver is given as a module, i.e. from firehose-core)
  • Add an execution timeout of 3 minutes per block by default (can be overriden in tier1/tier2 Configs) -- this is useful when an external (eth_call) is stuck on a forked block hash.
  • Revert 'initialBlocks' changes from v1.9.1 because a 'changing module hash' causes more trouble.
  • Wazero: bump v1.8.0 and activate caching of precompiled wasm modules in /tmp/wazero to decrease compilation time
  • Metering update: more detailed metering with addition of new metrics (live_uncompressed_read_bytes, live_uncompressed_read_forked_bytes, file_uncompressed_read_bytes, file_uncompressed_read_forked_bytes, file_compressed_read_forked_bytes, file_compressed_read_bytes, file_uncompressed_write_bytes, file_compressed_write_bytes). DEPRECATION WARNING: bytes_read and bytes_written metrics will be removed in the future, please use the new metrics for metering instead.
  • Manifest reader: increase timeout of remote spkg fetch to 5 minutes, up from 30 seconds

Client

  • Add substreams auth command, to authenticate via thegraph.market and to get a dev API Key.
  • Rename --discovery-endpoint into codegen-endpoint in substreams init command.
  • Add substreams codegen subgraph command that takes a substreams module and an spkg and that generates a simple subgraph from the module output.
  • On substreams init command, if flag --state-file is provided, the state file is used by default for project generation.
  • In substreams init command, the state file is named using a Date format and not using Unix anymore.
  • Tools->prometheus: added the possibility to override the start-block on an endpoint

v1.9.3

26 Jul 00:32
Compare
Choose a tag to compare
  • Fixed error handling issue in 'backprocessing' causing high CPU usage in tier1 servers
  • Fixed handling of packages referenced by ipfs:// URL (now simply using /api/v0/cat?arg=...)
  • Added --used-modules-only flag to substreams info to only show modules that are in execution tree for the given output_module

v1.9.2

18 Jul 18:09
Compare
Choose a tag to compare

Added

  • Added support for directly reading spkg file that is compressed with zstd (from http, gs, s3, azure or local)

Fixed

  • Prevent Noop handler from sending outputs with 'Stalled' step in cursor (which breaks substreams-sink-kv)

v1.9.1

16 Jul 17:59
Compare
Choose a tag to compare

Fixed

Fixed substreams hanging in production-mode on chains with a 'first-streamable-block' higher than 0:

  • all initialBlocks will be 'bumped' to the first-streamable-block if it is higher
  • this will affect the module hashes: use substreams info --first-streamable-block=<block_num> to see how a value will affect your modules
  • modules with initialBlocks higher than the first-streamable-block of a chain will be unaffected.

v1.9.0

11 Jul 01:31
Compare
Choose a tag to compare

Important BUG FIX

  • Fix a bug introduced in v1.6.0 that could result in corrupted store "state" file if all
    the "outputs" were already cached for a module in a given segment (rare occurence)
  • We recommend clearing your substreams cache after this upgrade and re-processing or
    validating your data if you use stores.

Fixed

  • substreams 'tools decode state' now correctly prints the kvops when pointing to store output files

Added

  • Expose a new intrinsic to modules: skip_empty_output, which causes the module output to be skipped if it has zero bytes. (Watch out, a protobuf object with all its default values will have zero bytes)
  • Improve schedule order (faster time to first block) for substreams with multiple stages when starting mid-chain