Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define protocol fee params and forward to driver #2098

Merged
merged 25 commits into from
Dec 12, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions crates/autopilot/src/arguments.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ use {
std::{
net::SocketAddr,
num::{NonZeroUsize, ParseFloatError},
str::FromStr,
time::Duration,
},
url::Url,
Expand Down Expand Up @@ -208,6 +209,11 @@ pub struct Arguments {
)]
pub solve_deadline: Duration,

/// A list of fee policies in the following format: `[LimitOrderOutOfMarket:0.1:0.2,LimitOrderInMarket::2.5]`
///
#[clap(long, env, use_value_delimiter = true)]
pub fee_policies: Vec<FeePolicy>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would it semantically mean to define multiple entries with the same policy but different parameters? That the protocol has multiple fees for the same order type? This sounds confusing to me.

I don't foresee us using vastly different parameters for in market vs out of market orders (if so we can define them later as two different fields). I therefore think we should just define a struct with

  • price improvement factor
  • volume % cap
  • absolute cap
  • whether or not to include in-market priced limit orders

On a per order basis, I think we should then just signal the concrete fee for this order, which can be one of three (maybe 4) in my opinion, potentially chained in case integrators want to charge additional fees.

  • % of price improvement (WE ONLY NEED THIS FOR NOW)
  • % of volume (e.g. 0.1% of volume)
  • Absolute amount (e.g. $1 per trade)
  • maybe % of surplus (although the incentives are too obviously bad to even experiment with this IMO)

Copy link
Contributor Author

@sunce86 sunce86 Dec 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would it semantically mean to define multiple entries with the same policy but different parameters?

It enables us to take a quote deviation fee, and after that another fee as percent of volume (meaning we can chain multiple different fees for a single order). IIUC, this is something we have to support for integrators.

One could argue that we should summarize all of them to a single value and send that single value to the driver, but can't do that since not all types of fees are known upfront (volume percent and absolute fee are, quote deviation not).

Then, since db migrations are painful, I decided to model the db table for fee polices in a way that supports all foreseeable types of policies and not just for price improvement type. That design was created to be memory efficient and decent performance wise (I could have created a separate table for each fee type but then the fetching would be slower).

Why

pub enum FeePolicy {
LimitOrderOutOfMarket(FeePolicyParams),
LimitOrderInMarket(FeePolicyParams),
}

Because I expect to add a third type - TWAP orders.

I hope my reasonings are clearer now. Note that this PR is not yet ready for merging, I just need to get the approval for the last commit that my new logic is ok.

Copy link
Contributor Author

@sunce86 sunce86 Dec 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I hoped that I could support these two types of fees in the first iteration, since it will be trivial with this design:

% of price improvement (WE ONLY NEED THIS FOR NOW)
% of volume (e.g. 0.1% of volume)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this is something we have to support for integrators.

My vision for integrators is that they specify their fee policy as part of the app data and it doesn't become part of the arguments we start the core protocol with.

The core protocol will experiment with different fees (price improvements, volume based, etc) but I foresee its fee config to be relatively concise and not a chain of many different fee compositions. Also, the protocol can only charge fees on order types that are a first class citizen in the core protocol (limit orders).

Because I expect to add a third type - TWAP orders.

TWAP orders are not part of the core CoW Protocol. They are a set of orders that semantically belong together and are orchestrated and placed from a watch tower, but as far as CoW protocol is concerned users are simply placing limit orders in awkwardly consistent periods.

So my goal here is to split what the protocol boots up with as a fee policy from the different types of protocol fees we can attach to each order that is sent to the solvers.

E.g. the protocol may define that we want to take 0.1% volume fee on all orders. This is what the autopilot should be started with.

If an integrator (e.g. CoW Swap's TWAP feature or Metamask) wants to charge 0.85% volume fee on top, then they can create an order specifying this in the app data and what we end up sending to the driver is two fees (0.1% for the protocol, 0.85% for the integrator), both of which will be charged from solvers if they settle this order at the end. Potentially the protocol will demand a cut (cf. Apple charging 30% of all app store purchases) from the integrator fees.

To sum up, I think we should have a FeePolicy struct on the autopilot arguments and the a protocol_fee: Vec<Fee> on the order itself.

Copy link
Contributor Author

@sunce86 sunce86 Dec 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, not using configuration type for autopilot domain makes sense, will split it.

Do you think we should keep the possibility that 2 different policies (for example quote deviation and volume based) can be sent for a single order to the driver, at the same time?

If you don't like having a third enum variant for TWAP orders, how would you model a situation where we set quote deviation policy for limit orders and volume based policy for TWAP orders using:

price improvement factor
volume % cap
absolute cap
whether or not to include in-market priced limit orders

We talked about not supporting this on slack, if so, I can just add another whether to include twap orders flag.


/// Time interval in days between each cleanup operation of the
/// `order_events` database table.
#[clap(long, env, default_value = "1", value_parser = duration_from_days)]
Expand Down Expand Up @@ -284,6 +290,7 @@ impl std::fmt::Display for Arguments {
writeln!(f, "score_cap: {}", self.score_cap)?;
display_option(f, "shadow", &self.shadow)?;
writeln!(f, "solve_deadline: {:?}", self.solve_deadline)?;
display_list(f, "fee_policies", self.fee_policies.iter())?;
writeln!(
f,
"order_events_cleanup_interval: {:?}",
Expand All @@ -298,6 +305,75 @@ impl std::fmt::Display for Arguments {
}
}

#[derive(clap::Parser, Clone)]
pub enum FeePolicy {
LimitOrderOutOfMarket(FeePolicyParams),
LimitOrderInMarket(FeePolicyParams),
}

impl FromStr for FeePolicy {
type Err = anyhow::Error;

fn from_str(s: &str) -> Result<Self, Self::Err> {
let mut parts = s.split(':');
let policy = parts.next().unwrap();
let params = FeePolicyParams {
price_improvement_factor: parts.next().unwrap().parse().ok(),
volume_factor: parts.next().unwrap().parse().ok(),
};
match policy {
"LimitOrderOutOfMarket" => Ok(FeePolicy::LimitOrderOutOfMarket(params)),
"LimitOrderInMarket" => Ok(FeePolicy::LimitOrderInMarket(params)),
_ => Err(anyhow::anyhow!("Unknown fee policy: {}", policy)),
}
}
}

impl std::fmt::Display for FeePolicy {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
FeePolicy::LimitOrderOutOfMarket(params) => {
write!(f, "LimitOrderOutOfMarket {{ {} }}", params)
}
FeePolicy::LimitOrderInMarket(params) => {
write!(f, "LimitOrderInMarket {{ {} }}", params)
}
}
}
}

#[derive(clap::Parser, Clone)]
pub struct FeePolicyParams {
/// How much of the order's price improvement over max(limit price,
/// best_bid) should be taken as a protocol fee.
#[clap(
long,
env,
default_value = None,
value_parser = shared::arguments::parse_percentage_factor
)]
pub price_improvement_factor: Option<f64>,
sunce86 marked this conversation as resolved.
Show resolved Hide resolved
/// How much of the order's volume should be taken as a protocol fee.
#[clap(
long,
env,
default_value = None,
value_parser = shared::arguments::parse_percentage_factor
)]
pub volume_factor: Option<f64>,
fleupold marked this conversation as resolved.
Show resolved Hide resolved
}

impl std::fmt::Display for FeePolicyParams {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
writeln!(
f,
"price_improvement_factor: {:?}, volume_factor: {:?}",
self.price_improvement_factor, self.volume_factor
)?;
Ok(())
}
}

fn duration_from_days(s: &str) -> Result<Duration, ParseFloatError> {
let days = s.parse::<f64>()?;
Ok(Duration::from_secs_f64(days * 86_400.0))
Expand Down
28 changes: 28 additions & 0 deletions crates/autopilot/src/driver_model.rs
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ pub mod solve {
pub app_data: AppDataHash,
#[serde(flatten)]
pub signature: Signature,
/// The types of fees that should be collected by the protocol.
/// The driver is expected to apply the fees in the order they are
/// listed.
sunce86 marked this conversation as resolved.
Show resolved Hide resolved
pub fee_policies: Vec<FeePolicy>,
}

#[derive(Clone, Debug, Serialize, Deserialize)]
Expand Down Expand Up @@ -150,6 +154,30 @@ pub mod solve {
pub submission_address: H160,
}

#[derive(Clone, Debug, Serialize, Deserialize)]
sunce86 marked this conversation as resolved.
Show resolved Hide resolved
#[serde(rename_all = "camelCase")]
pub enum FeePolicy {
/// Applies to limit orders only.
/// This fee should be taken if the solver provided good enough solution
/// that even after the surplus fee is taken, there is still more
/// surplus left above whatever the user expects [order limit price
/// or best quote, whichever is better for the user].
sunce86 marked this conversation as resolved.
Show resolved Hide resolved
QuoteDeviation {
sunce86 marked this conversation as resolved.
Show resolved Hide resolved
/// Percentage of the order's `available surplus` should be taken as
/// a protocol fee.
///
/// `Available surplus` is the difference between the executed_price
/// (adjusted by surplus_fee) and the closer of the two: order
/// limit_price or best_quote. For out-of-market limit orders,
/// order limit price is closer to the executed price. For
/// in-market limit orders, best quote is closer to the executed
/// price.
sunce86 marked this conversation as resolved.
Show resolved Hide resolved
factor: f64,
/// Cap protocol fee with a percentage of the order's volume.
volume_cap_factor: f64,
},
}

#[derive(Clone, Debug, Default, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct Response {
Expand Down
2 changes: 2 additions & 0 deletions crates/autopilot/src/run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,7 @@ pub async fn run(args: Arguments) {
score_cap: args.score_cap,
max_settlement_transaction_wait: args.max_settlement_transaction_wait,
solve_deadline: args.solve_deadline,
fee_policies: args.fee_policies,
};
run.run_forever().await;
unreachable!("run loop exited");
Expand Down Expand Up @@ -691,6 +692,7 @@ async fn shadow_mode(args: Arguments) -> ! {
trusted_tokens,
args.score_cap,
args.solve_deadline,
args.fee_policies,
);
shadow.run_forever().await;

Expand Down
40 changes: 39 additions & 1 deletion crates/autopilot/src/run_loop.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
use {
crate::{
arguments,
database::{
competition::{Competition, ExecutedFee, OrderExecution},
Postgres,
Expand All @@ -8,7 +9,7 @@ use {
driver_model::{
reveal::{self, Request},
settle,
solve::{self, Class},
solve::{self, Class, FeePolicy},
},
solvable_orders::SolvableOrdersCache,
},
Expand Down Expand Up @@ -54,6 +55,7 @@ pub struct RunLoop {
pub score_cap: U256,
pub max_settlement_transaction_wait: Duration,
pub solve_deadline: Duration,
pub fee_policies: Vec<arguments::FeePolicy>,
}

impl RunLoop {
Expand Down Expand Up @@ -302,6 +304,7 @@ impl RunLoop {
&self.market_makable_token_list.all(),
self.score_cap,
self.solve_deadline,
self.fee_policies.clone(),
);
let request = &request;

Expand Down Expand Up @@ -449,6 +452,7 @@ pub fn solve_request(
trusted_tokens: &HashSet<H160>,
score_cap: U256,
time_limit: Duration,
fee_policies: Vec<arguments::FeePolicy>,
) -> solve::Request {
solve::Request {
id,
Expand All @@ -474,6 +478,39 @@ pub fn solve_request(
.collect()
};
let order_is_untouched = remaining_order.executed_amount.is_zero();
let fee_policies = fee_policies
.iter()
.filter_map(|policy| match policy {
arguments::FeePolicy::LimitOrderInMarket(params) => {
tracing::warn!("LimitOrderInMarket fee policy not yet supported");
None
}
arguments::FeePolicy::LimitOrderOutOfMarket(params) => {
match params.price_improvement_factor {
Some(price_improvement_factor) => {
// todo
None
}
None => {
tracing::warn!("not supported yet");
None
}
}
}
})
.collect();

// let fee_policies = match order.metadata.class {
// OrderClass::Market => vec![],
// OrderClass::Liquidity => vec![],
// // todo https://github.com/cowprotocol/services/issues/2092
// // skip protocol fee for limit orders with in-market price
// // OrderClass::Limit(_) => vec![FeePolicy::QuoteDeviation {
// // factor: quote_deviation_policy.protocol_fee_factor,
// // volume_cap_factor:
// quote_deviation_policy.protocol_fee_volume_cap_factor, //
// }], OrderClass::Limit(_) => vec![],
// };
solve::Order {
uid: order.metadata.uid,
sell_token: order.data.sell_token,
Expand All @@ -499,6 +536,7 @@ pub fn solve_request(
class,
app_data: order.data.app_data,
signature: order.signature.clone(),
fee_policies,
}
})
.collect(),
Expand Down
10 changes: 9 additions & 1 deletion crates/autopilot/src/shadow.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,12 @@

use {
crate::{
arguments::FeePolicy,
driver_api::Driver,
driver_model::{reveal, solve},
driver_model::{
reveal,
solve::{self},
},
protocol,
run_loop,
},
Expand Down Expand Up @@ -43,6 +47,7 @@ pub struct RunLoop {
block: u64,
score_cap: U256,
solve_deadline: Duration,
fee_policies: Vec<FeePolicy>,
}

impl RunLoop {
Expand All @@ -52,6 +57,7 @@ impl RunLoop {
trusted_tokens: AutoUpdatingTokenList,
score_cap: U256,
solve_deadline: Duration,
fee_policies: Vec<FeePolicy>,
) -> Self {
Self {
orderbook,
Expand All @@ -61,6 +67,7 @@ impl RunLoop {
block: 0,
score_cap,
solve_deadline,
fee_policies,
}
}

Expand Down Expand Up @@ -193,6 +200,7 @@ impl RunLoop {
&self.trusted_tokens.all(),
self.score_cap,
self.solve_deadline,
self.fee_policies.clone(),
);
let request = &request;

Expand Down
22 changes: 22 additions & 0 deletions crates/driver/src/domain/competition/order/fees.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#[derive(Clone, Debug)]
pub enum FeePolicy {
/// Applies to limit orders only.
/// This fee should be taken if the solver provided good enough solution
/// that even after the surplus fee is taken, there is still more
/// surplus left above whatever the user expects [order limit price
/// vs best quote].
QuoteDeviation {
/// Percentage of the order's `available surplus` should be taken as a
/// protocol fee.
///
/// `Available surplus` is the difference between the executed_price
/// (adjusted by surplus_fee) and the closer of the two: order
/// limit_price or best_quote. For out-of-market limit orders,
/// order limit price is closer to the executed price. For
/// in-market limit orders, best quote is closer to the executed
/// price.
factor: f64,
/// Cap protocol fee with a percentage of the order's volume.
volume_cap_factor: f64,
},
}
7 changes: 6 additions & 1 deletion crates/driver/src/domain/competition/order/mod.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
pub use signature::Signature;
use {
super::auction,
crate::{
Expand All @@ -9,7 +8,9 @@ use {
bigdecimal::Zero,
num::CheckedDiv,
};
pub use {fees::FeePolicy, signature::Signature};

pub mod fees;
pub mod signature;

/// An order in the auction.
Expand Down Expand Up @@ -39,6 +40,9 @@ pub struct Order {
pub sell_token_balance: SellTokenBalance,
pub buy_token_balance: BuyTokenBalance,
pub signature: Signature,
/// The types of fees that should be collected by the protocol.
/// The driver is expected to apply the fees in the order they are listed.
pub fee_policies: Vec<FeePolicy>,
sunce86 marked this conversation as resolved.
Show resolved Hide resolved
}

/// An amount denominated in the sell token of an [`Order`].
Expand Down Expand Up @@ -453,6 +457,7 @@ mod tests {
data: Default::default(),
signer: Default::default(),
},
fee_policies: Default::default(),
};

assert_eq!(
Expand Down
1 change: 1 addition & 0 deletions crates/driver/src/domain/quote.rs
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ impl Order {
data: Default::default(),
signer: Default::default(),
},
fee_policies: Default::default(),
}],
[
auction::Token {
Expand Down
20 changes: 20 additions & 0 deletions crates/driver/src/infra/api/routes/solve/dto/auction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,19 @@ impl Auction {
data: order.signature.into(),
signer: order.owner.into(),
},
fee_policies: order
.fee_policies
.into_iter()
.map(|policy| match policy {
FeePolicy::QuoteDeviation {
factor,
volume_cap_factor,
} => competition::order::FeePolicy::QuoteDeviation {
factor,
volume_cap_factor,
},
})
.collect(),
})
.collect(),
self.tokens.into_iter().map(|token| {
Expand Down Expand Up @@ -234,6 +247,7 @@ struct Order {
signing_scheme: SigningScheme,
#[serde_as(as = "serialize::Hex")]
signature: Vec<u8>,
fee_policies: Vec<FeePolicy>,
}

#[derive(Debug, Deserialize)]
Expand Down Expand Up @@ -287,3 +301,9 @@ enum Class {
Limit,
Liquidity,
}

#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
enum FeePolicy {
QuoteDeviation { factor: f64, volume_cap_factor: f64 },
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the dto also have documentation (so that it can be generated for the swagger docs)?

Loading
Loading