Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch insertion #259

Closed
recmo opened this issue Nov 2, 2022 · 1 comment · Fixed by #386
Closed

Batch insertion #259

recmo opened this issue Nov 2, 2022 · 1 comment · Fixed by #386
Milestone

Comments

@recmo
Copy link
Contributor

recmo commented Nov 2, 2022

Requires #258.

Modify the transaction worker service (B). From it's #258 behaviour:

  1. It takes an item from the work-queue.
  2. It flags it as in-progress.
  3. It creates and signs an Ethereum transaction.
  4. It waits for the transaction to be mined.
  5. Marks the work item as done.

Into the following:


  1. Check the work-queue. If it has more than MAX_BATCH unprocessed items or the oldest unprocessed item is older than MAX_DELAY, proceed.
  2. Take up to MAX_BATCH unprocessed items from work queue, oldest first.
  3. Mark these as 'processing' with a timestamp.
  4. Request next available nonce for the transaction, and flag this nonce as taken.
  5. Send the batch to a prover.
  6. When the proof is done, create and sign a transaction.
  7. Send transaction to mempool using one or more API endpoints.
  8. Babysit the transaction, re-send and/or raise gasprice if necessary.
  9. Once the transaction is mined and sufficiently confirmed, mark work items as done.

TBD:

  • Create a ZKP prover services for the compute-heavy proving tasks
  • Mechanism for recovering from failed work. For example a timeout based on the timestamp in 3.
  • Do we want to split transaction signing/sending to a separate service? It can be a reusable component.
@recmo recmo added this to the Launch milestone Nov 2, 2022
@kustosz
Copy link
Contributor

kustosz commented Nov 21, 2022

Re: "ZKP prover services for the compute-heavy proving tasks" – this is now tracked in worldcoin/semaphore-mtb#3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants