Add federation sync configuration SQL table and RPC endpoints #537

ffranr · 2023-09-28T16:27:31Z

This PR adds functionality to configure the universe federation sync routine. It facilitates both a general configuration and a universe/asset specific configuration.

This PR is part of the work necessary to complete #530 .

This change is

tapdb/sqlc/migrations/000007_universe.up.sql

universe/interface.go

tapdb/sqlc/migrations/000007_universe.up.sql

tapdb/universe_federation.go

rpcserver.go

ffranr · 2023-10-03T01:58:41Z

All set/query db transactions are now atomic.

I've added an enum to signify which proofs should be synced. I think it's the cleanest solution because each sync process will either exclude proofs, sync issuance, or sync all proofs. We can also expand the enum (which comes close to Oli's key-value idea.)

Let me know what you guys think @guggero @Roasbeef

universe/interface.go

tapdb/sqlc/migrations/000007_universe.up.sql

tapdb/universe_federation.go

rpcserver.go

taprpc/universerpc/universe.proto

guggero

Looks pretty good. I think the level of configurability makes sense for the moment.
I assume the code to actually sync according to the setting will be done in a follow-up PR?

universe/interface.go

tapdb/universe_federation.go

rpcserver.go

taprpc/universerpc/universe.proto

guggero · 2023-10-04T08:36:21Z

tapdb/universe_federation.go

+
+// QueryFederationSyncConfigs returns the general and universe specific
+// federation sync configs.
+func (u *UniverseFederationDB) QueryFederationSyncConfigs(


A standalone unit test for DB operations would be nice.

taprpc/universerpc/universe.proto

itest/universe_test.go

Roasbeef · 2023-10-04T22:12:35Z

I assume the code to actually sync according to the setting will be done in a follow-up PR?

Yep plan is to do this as we work through #544.

Roasbeef · 2023-10-04T22:12:40Z

Needs a rebase!

guggero

LGTM 🎉

tapdb/sqlc/migrations/000007_universe.down.sql

tapdb/universe_federation.go

This commit adds two tables for storing universe federation sync configuration settings: 1. federation_global_sync_config stores global (but proof type specific) federation sync configuration. 2. federation_uni_sync_config stores universe specific sync configuration.

Add RPC endpoints for setting and querying for federation sync config: * SetFederationSyncConfig * QueryFederationSyncConfig

Roasbeef

Reviewed 3 of 16 files at r5, all commit messages.
Reviewable status: 9 of 22 files reviewed, 37 unresolved discussions (waiting on @ffranr, @guggero, and @jharveyb)

rpcserver.go line 2598 at r4 (raw file):

	return &universe.FedUniSyncConfig{
		UniverseID: uniID,
		ProofTypes: proofTypes,

I think we still want this here to distinguish between issuance/transfer trees for insert/export?

Roasbeef

LGTM 🧲

Roasbeef · 2023-10-11T02:41:20Z

tapdb/sqlc/migrations/000007_universe.up.sql

+
+-- This table contains universe (asset/asset group) specific federation sync
+-- configuration.
+CREATE TABLE IF NOT EXISTS federation_uni_sync_config (


IIUC, in order for something to later on have a foreign key that refs this table, it does in fact still need a primary key.

I think we just want a composite primary key here. Though I saw you had to re-work things slightly tho, non-blocking comment.

ffranr requested review from Roasbeef and jharveyb September 28, 2023 16:27

ffranr force-pushed the federation-sync-config branch from 43591a1 to fd02dbd Compare September 28, 2023 16:35

guggero reviewed Sep 29, 2023

View reviewed changes

tapdb/sqlc/migrations/000007_universe.up.sql Outdated Show resolved Hide resolved

ffranr force-pushed the federation-sync-config branch 4 times, most recently from 010689b to 76571f6 Compare September 29, 2023 14:56

Roasbeef requested changes Sep 29, 2023

View reviewed changes

ffranr force-pushed the federation-sync-config branch from 76571f6 to 4f42c0a Compare October 3, 2023 01:54

ffranr requested review from guggero and Roasbeef October 3, 2023 01:58

ffranr force-pushed the federation-sync-config branch 3 times, most recently from eacca37 to f38d319 Compare October 3, 2023 12:24

Roasbeef requested changes Oct 4, 2023

View reviewed changes

guggero reviewed Oct 4, 2023

View reviewed changes

ffranr force-pushed the federation-sync-config branch 4 times, most recently from e037319 to e05294b Compare October 4, 2023 15:14

ffranr force-pushed the federation-sync-config branch from e05294b to 2af624b Compare October 5, 2023 15:23

ffranr changed the base branch from main to uni-namespace-proof-type October 5, 2023 15:23

ffranr force-pushed the uni-namespace-proof-type branch from 7b813c1 to 5557ce6 Compare October 5, 2023 15:52

ffranr self-assigned this Oct 6, 2023

ffranr force-pushed the uni-namespace-proof-type branch from 5557ce6 to 05da767 Compare October 6, 2023 15:01

Roasbeef requested a review from guggero October 7, 2023 00:46

ffranr force-pushed the uni-namespace-proof-type branch from b1c6588 to 3ab1c01 Compare October 9, 2023 21:32

ffranr force-pushed the federation-sync-config branch 3 times, most recently from 54f716f to 0dcd95c Compare October 10, 2023 01:07

ffranr requested review from Roasbeef and guggero October 10, 2023 01:10

guggero approved these changes Oct 10, 2023

View reviewed changes

tapdb/sqlc/migrations/000007_universe.down.sql Outdated Show resolved Hide resolved

ffranr force-pushed the federation-sync-config branch 2 times, most recently from fe93d2c to 7f6d3fa Compare October 10, 2023 19:53

Roasbeef reviewed Oct 10, 2023

View reviewed changes

tapdb/universe_federation.go Outdated Show resolved Hide resolved

ffranr force-pushed the federation-sync-config branch 5 times, most recently from 8544884 to 3616bd6 Compare October 10, 2023 23:36

ffranr added 4 commits October 11, 2023 00:56

rpc: add query and set RPC endpoints for federation sync config

8025ecb

Add RPC endpoints for setting and querying for federation sync config: * SetFederationSyncConfig * QueryFederationSyncConfig

rpc: export function unmarshall universe ID

9dd3f46

itest: add test to check set/query for federation sync config

95b03be

ffranr force-pushed the federation-sync-config branch from 3616bd6 to 95b03be Compare October 10, 2023 23:57

ffranr requested a review from Roasbeef October 11, 2023 00:22

ffranr mentioned this pull request Oct 11, 2023

Add proof type to universe identifer #540

Merged

Roasbeef requested a review from guggero October 11, 2023 02:37

Roasbeef reviewed Oct 11, 2023

View reviewed changes

Roasbeef approved these changes Oct 11, 2023

View reviewed changes

Roasbeef merged commit 28e4a6b into uni-namespace-proof-type Oct 11, 2023
14 checks passed

guggero deleted the federation-sync-config branch October 11, 2023 07:24

ffranr restored the federation-sync-config branch October 11, 2023 11:59

ffranr deleted the federation-sync-config branch October 11, 2023 12:28

ffranr mentioned this pull request Oct 11, 2023

Federation sync config #563

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add federation sync configuration SQL table and RPC endpoints #537

Add federation sync configuration SQL table and RPC endpoints #537

ffranr commented Sep 28, 2023 •

edited by Roasbeef

Loading

ffranr commented Oct 3, 2023

guggero left a comment

guggero Oct 4, 2023

Roasbeef commented Oct 4, 2023

Roasbeef commented Oct 4, 2023

guggero left a comment

Roasbeef left a comment

Roasbeef left a comment

Roasbeef Oct 11, 2023

Add federation sync configuration SQL table and RPC endpoints #537

Add federation sync configuration SQL table and RPC endpoints #537

Conversation

ffranr commented Sep 28, 2023 • edited by Roasbeef Loading

ffranr commented Oct 3, 2023

guggero left a comment

Choose a reason for hiding this comment

guggero Oct 4, 2023

Choose a reason for hiding this comment

Roasbeef commented Oct 4, 2023

Roasbeef commented Oct 4, 2023

guggero left a comment

Choose a reason for hiding this comment

Roasbeef left a comment

Choose a reason for hiding this comment

Roasbeef left a comment

Choose a reason for hiding this comment

Roasbeef Oct 11, 2023

Choose a reason for hiding this comment

ffranr commented Sep 28, 2023 •

edited by Roasbeef

Loading