Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add is_winner to solver competition endpoints #3127

Merged

Conversation

m-lord-renkse
Copy link
Contributor

Description

Add is_winner to solver competition endpoints

Changes

Add is_winner boolean to each solution json object.

How to test

  1. e2e test
  2. Regression test

Related Issues

Fixes #3070

@m-lord-renkse m-lord-renkse force-pushed the 3070/add-is-winer-to-solver-competition-endpoint branch from c7cde9c to 4272ac4 Compare November 14, 2024 15:45
@m-lord-renkse m-lord-renkse force-pushed the 3070/add-is-winer-to-solver-competition-endpoint branch from 4272ac4 to fcf323b Compare November 15, 2024 08:58
@m-lord-renkse m-lord-renkse marked this pull request as ready for review November 15, 2024 09:03
@m-lord-renkse m-lord-renkse requested a review from a team as a code owner November 15, 2024 09:03
Copy link
Contributor

@sunce86 sunce86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are missing some kind of migration to populate the value for historic entries - otherwise those will be false for all solutions in an auction.

This migration can be done manually after this PR is merged as well. Simply, go over entries in solver_competition and set is_winner=true for all last solutions.

@@ -54,6 +54,7 @@ pub struct SolverSettlement {
#[serde_as(as = "BTreeMap<_, HexOrDecimalU256>")]
pub clearing_prices: BTreeMap<H160, U256>,
pub orders: Vec<Order>,
pub is_winner: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add #[serde(default)] above the new field to make it explicit that is_winner might be missing for some entries and to make the deserialization more robust.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just do the migration in this PR as well to have al the data correct in one go?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MartinquaXD see my message below: #3127 (comment)

I would love to hear your opinion. I am open to do a migration in this PR, but I assume a SQL migration is off the table due to the json characteristics 🤔

crates/orderbook/openapi.yml Outdated Show resolved Hide resolved
Copy link

This pull request has been marked as stale because it has been inactive a while. Please update this pull request or it will be automatically closed.

@squadgazzz
Copy link
Contributor

Can this PR be marked as Draft to avoid redundant notifications until all the comments are addressed?

@m-lord-renkse m-lord-renkse marked this pull request as draft November 26, 2024 08:08
Copy link

github-actions bot commented Dec 2, 2024

Reminder: Please consider backward compatibility when modifying the API specification.
If breaking changes are unavoidable, ensure:

  • You explicitly pointed out breaking changes.
  • You communicate the changes to affected teams.
  • You provide proper versioning and migration mechanisms.

Caused by:

@m-lord-renkse
Copy link
Contributor Author

I think we are missing some kind of migration to populate the value for historic entries - otherwise those will be false for all solutions in an auction.

This migration can be done manually after this PR is merged as well. Simply, go over entries in solver_competition and set is_winner=true for all last solutions.

hmm I don't really like these type of migrations on tables which are just json, because an undetected error can be fatal 😅 have we considered moving this to a normal table, or at least move the json to a versioned json?

With a versioned json we can have a custom deserialization, and it will deserialize back according to the json version stored.

If this is not an option, how do we usually do these type of migrations? this would need a binary.

@m-lord-renkse m-lord-renkse marked this pull request as ready for review December 3, 2024 14:48
@m-lord-renkse m-lord-renkse requested a review from sunce86 December 3, 2024 14:48
Copy link
Contributor

@mstrug mstrug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok.

crates/orderbook/openapi.yml Show resolved Hide resolved
crates/orderbook/openapi.yml Outdated Show resolved Hide resolved
@@ -54,6 +54,7 @@ pub struct SolverSettlement {
#[serde_as(as = "BTreeMap<_, HexOrDecimalU256>")]
pub clearing_prices: BTreeMap<H160, U256>,
pub orders: Vec<Order>,
pub is_winner: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just do the migration in this PR as well to have al the data correct in one go?

@MartinquaXD
Copy link
Contributor

an undetected error can be fatal

In this specific case it should be fairly straight forward to test.

have we considered moving this to a normal table

Actually yes. @sunce86 did a bunch of refactoring work to store this information in regular tables. I'm not why the need of adding this suddenly arose but technically we could also focus on reading the relevant data from the new tables and only add the value there.
@sunce86 is this still accurate?

With a versioned json we can have a custom deserialization, and it will deserialize back according to the json version stored.
If this is not an option, how do we usually do these type of migrations? this would need a binary.

So far we didn't have tables which needed to be deserialized from different formats. Meaning we basically forwarded the JSON inside the column as is and didn't parse it in the backend after reading it out.

If we actually require more guarantees on the stability of the format versioning would be a good idea. 👍

@sunce86
Copy link
Contributor

sunce86 commented Dec 4, 2024

Yes, we have is_winner field in proposed_solutions table that is supposed to replace solver_competition table. I think the most important part of this PR is to add is_winner to API struct for solver competition endpoint (in model crate) and that is done.

Actual populating of historic data for proposed_solutions is captured in #3056 and once it's done we can implement reading solver competition data from it and drop solver_competition table.

Important notice is that after this PR we wont have properly populated is_winner in solver_competition table for historic entries. But considering that solver_competition table will be dropped, I'm fine with it.

Copy link
Contributor

@sunce86 sunce86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG. Let's just not consider the feature finished (announce it to other teams and solvers) since we still need to take care of the historic entries I explained ☝️ )

@m-lord-renkse
Copy link
Contributor Author

Pending to merge until @anxolin gives me the green light.

@m-lord-renkse m-lord-renkse merged commit c47c80c into main Dec 9, 2024
11 checks passed
@m-lord-renkse m-lord-renkse deleted the 3070/add-is-winer-to-solver-competition-endpoint branch December 9, 2024 09:57
@github-actions github-actions bot locked and limited conversation to collaborators Dec 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: Add is_winner flag to solver competition endpoint
5 participants