Merge branch 'streamingfast:develop' into develop

antelope-common/README.md

@@ -25,7 +25,7 @@ Common use cases:
 Let's say you want to receive all AtomicAssets NFT create collection events starting from block 370,000,000.
 Send a substreams request with the desired query as parameter. You can use a gRPC client, substreams sink, or substreams CLI:
 ```bash
-> substreams gui -e eos.substreams.pinax.network:443 https://spkg.io/pinax-network/antelope-common-v0.2.0.spkg filtered_actions -s 370000000 -p filtered_actions="code:atomicassets && action:createcol" --production-mode
+> substreams gui -e eos.substreams.pinax.network:443 https://spkg.io/pinax-network/antelope-common-v0.3.0.spkg filtered_actions -s 370000000 -p filtered_actions="code:atomicassets && action:createcol" --production-mode
 ```
 If the request with this query hasn't been run before, substreams backend will start the indexing process and you should start seeing new events. If the request has been run before, you should start seeing sale actions right away jumping over any empty chunks of blocks when there were no sales.
 Note, we used `--production-mode` flag - this ensures backend writes indexes on disk so they can be re-used in the future. 
@@ -59,7 +59,7 @@ v0.3.0: https://substreams.dev/pinax-network/antelope-common/v0.3.0
 
 ### Usage
 ```bash
-substreams gui -e eos.substreams.pinax.network:443 https://spkg.io/pinax-network/antelope-common-v0.2.0.spkg filtered_actions -s -10000 -p filtered_actions="code:tethertether && data.to:swap.defi" --production-mode
+substreams gui -e eos.substreams.pinax.network:443 https://spkg.io/pinax-network/antelope-common-v0.3.0.spkg filtered_actions -s -10000 -p filtered_actions="code:tethertether && data.to:swap.defi" --production-mode
 ```
 
 ### Known issues

injective-common/Cargo.toml

@@ -14,7 +14,7 @@ strip = "debuginfo"
 crate-type = ["cdylib"]
 
 [dependencies]
-substreams = "^0.5.19"
+substreams = "^0.5.20"
 hex = "0.4.3"
 prost = "0.11"
 prost-types = "0.11"

injective-common/README.md

@@ -2,34 +2,21 @@
 
 The Injective Foundational modules are Substreams modules extracting common data from the Injective blockchain.
 
+## Getting the substreams package
 
+### From Substreams.dev
 
-## Modules
-
-### all_events
-
-The `all_events` module extracts only the events and provides them, along with the transaction hash and block at which it they are found.
+https://substreams.dev/streamingfast/injective-common/
 
-### filtered_events
 
-The `filtered_events` module allows a reduction of the `all_events` output, only matching the events that match the requested type.
+### Building it yourself
 
-Use with the parameter, ex: `substreams run [...] -p filtered_events="message || injective.peggy.v1.EventDepositClaim"
+### Protobuf definitions bundle
 
-### all_transactions (work in progress)
+The required protobuf modules are referenced in `buf.yaml` and bundled in `generated-buf-build.binpb`
 
-The `all_transactions` module extracts all the transactions from the Injective blockchain, providing useful information, such as _messages_, _signatures_ or _events_.
-
-Some message types are parsed from their "Any" type into the the corresponding type of an enum. See ./proto/cosmos/v1/transactions.proto to see the ones that are supported.
-The other types will still be shown as protobuf "Any" type.
-
-## Getting Started
-
-### Gather protobuf definitions in generated-buf-build.binpb 
-
-The required protobuf modules are referenced in `buf.yaml`. 
-You need a (free) API token to access https://buf.build and resolve the dependencies into a single file, generated-buf-build.binpb.
-That file is then used to generate the rust protobuf bindings or to bundle the definitions in the .spkg. (it is referenced in the substreams.yaml)
+If you want to add more, you will need a (free) API token to access https://buf.build
+The `make` command will automatically run the appropriate `buf build` command if needed.
 
 ### Generate rust protobuf bindings
 
@@ -47,14 +34,96 @@ make build
 
 You will need an API key to access the streamingfast servers, see https://substreams.streamingfast.io
 
-This example query only fetches the events of type 'injective.peggy.v1.EventDepositClaim'
-
 ```bash
-substreams run -e mainnet.injective.streamingfast.io:443 substreams.yaml filtered_events -p filtered_events='injective.peggy.v1.EventDepositClaim' -s 9600 -t 9700
+substreams run -e mainnet.injective.streamingfast.io:443 substreams.yaml filtered_events -p filtered_events='type:injective.peggy.v1.EventDepositClaim' -s 9600 -t 9700
+substreams run -e testnet.injective.streamingfast.io:443 substreams.yaml filtered_events -p filtered_events='type:injective.exchange.v1beta1.EventCancelSpotOrder' -s 27751658 -t +10
 ```
 
-### Package as .spkg
+### Package it as .spkg
+
+* optionally bump the version number in substreams.yaml
 
 ```bash
 make package
 ```
+
+## Modules
+
+### all_events
+
+The `all_events` module reads from the `sf.cosmos.type.v2.Block` source and outputs a list of all events in the block,
+Some events are at the block level, others appear inside the transactions. The latter will include the transaction hash.
+
+### filtered_events
+
+The `filtered_events` module allows a reduction of the `all_events` output, only matching the events that match the 
+requested type or include the attribute keys.
+It pre-filters blocks using the `index_events` module for efficiency.
+
+Use with parameters, ex:
+
+```bash
+substreams run [...] -p filtered_events="(type:message && attr:action) || (type:wasm && attr:_contract_address)"
+```
+
+### filtered_event_groups
+
+The `filtered_event_groups` module reads from `all_events` and applies a filter on the event types and attribute keys, 
+outputting *all the events* from transactions that have at least one event matching the filter.
+It pre-filters blocks using the `index_events` module for efficiency.
+
+Use with the same parameters as the `filtered_events` module.
+
+### all_transactions
+
+The `all_transactions` module extracts all the transactions from the Injective blockchain, providing useful information, such as _messages_, _signatures_ or _events_.
+
+Some message types are parsed from their "Any" type into the the corresponding type of an enum. See ./proto/cosmos/v1/transactions.proto to see the ones that are supported.
+The other types will still be shown as protobuf "Any" type.
+
+### filtered_trx_by_events
+
+
+The `filtered_trx_by_events` modules reads from `all_transactions` and applies a filter on the event types and attribute keys, 
+outputting the complete transactions that have at least one event matching the filter.
+It pre-filters blocks using the `index_events` module for efficiency.
+
+Use with the same parameters as the `filtered_events` module.
+
+
+### filtered_events_by_attribute_value
+
+The `filtered_events_by_attribute_value` module reads from `all_events` and applies a filter on the event types, 
+attribute keys and values, only outputting the events that match the filter. 
+
+Use with parameters, ex:
+
+```bash
+substreams run [...] -p filtered_events_by_attribute_value="type:wasm && attr:_contract_address:inj1v77y5ttah96dc9qkcpc88ad7rce8n88e99t3m5"
+```
+
+For an optimized experience, we recommend importing this module in your own substreams.yaml, overloading 
+the 'blockFilter' with a custom query to `index_events`, with the `use` parameter. See [an example](./derived-substreams-example.yaml)
+
+
+### filtered_event_groups_by_attribute_value
+
+The `filtered_events_groups_by_attribute_value` module reads from `all_events` and applies a filter on the event types,
+attribute keys and values, outputting all the events from transactions that have at least one event matching the filter.
+
+Use with the same parameters as the `filtered_events_by_attribute_value` module.
+
+For an optimized experience, we recommend importing this module in your own substreams.yaml, overloading 
+the 'blockFilter' with a custom query to `index_events`, with the `use` parameter. See [an example](./derived-substreams-example.yaml)
+
+
+### filtered_trx_by_events_attribute_value
+
+The `filtered_trx_by_events_attribute_value` module reads from `all_events` and applies a filter on the event types, 
+attribute keys and values, outputting the complete transactions that have at least one event matching the filter.
+
+Use with the same parameters as the `filtered_events_by_attribute_value` module.
+
+For an optimized experience, we recommend importing this module in your own substreams.yaml, overloading 
+the 'blockFilter' with a custom query to `index_events`, with the `use` parameter. See [an example](./derived-substreams-example.yaml)
+

injective-common/derived-substreams-example.yaml

@@ -0,0 +1,43 @@
+# requires substreams v1.7.1 and above
+specVersion: v0.1.0
+package:
+  name: example
+  version: v0.1.0
+
+network: injective-mainnet
+
+imports:
+  injective: ./injective-common-v0.2.1.spkg
+
+modules:
+  - name: my_events
+    use: injective:filtered_events_by_attribute_value
+    initialBlock: 70000000
+    blockFilter:
+      module: injective:index_events
+      query:
+        string: type:wasm && attr:_contract_address
+
+  - name: my_event_groups
+    use: injective:filtered_event_groups_by_attribute_value
+    initialBlock: 70000000
+    blockFilter:
+      module: injective:index_events
+      query:
+        string: type:wasm && attr:_contract_address
+
+  - name: my_transactions
+    use: injective:filtered_trx_by_events_attribute_value
+    initialBlock: 70000000
+    blockFilter:
+      module: injective:index_events
+      query:
+        string: type:wasm && attr:_contract_address
+
+params:
+  my_events: "type:wasm && attr:_contract_address:inj1v77y5ttah96dc9qkcpc88ad7rce8n88e99t3m5"
+  #my_events: "type:wasm && attr:_contract_address:inj1ja2z7lhtpq7myj9vv936euwepzw9hppd2fd5t0"
+  my_event_groups: "type:wasm && attr:_contract_address:inj1v77y5ttah96dc9qkcpc88ad7rce8n88e99t3m5"
+  #my_event_groups: "type:wasm && attr:_contract_address:inj1ja2z7lhtpq7myj9vv936euwepzw9hppd2fd5t0"
+  my_transactions: "type:wasm && attr:_contract_address:inj1v77y5ttah96dc9qkcpc88ad7rce8n88e99t3m5"
+  #my_transactions: "type:wasm && attr:_contract_address:inj1ja2z7lhtpq7myj9vv936euwepzw9hppd2fd5t0"