forked from b-inary/postflop-solver
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
10 changed files
with
588 additions
and
82 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,200 @@ | ||
# Design | ||
|
||
This document is a description, as far as I understand it, of the inner design | ||
of the solver and PostFlopGame. This is a working document for me to get my | ||
bearings. | ||
|
||
## PostFlopGame | ||
|
||
### Build/Allocate/Initialize | ||
|
||
To set up a `PostFlopGame` we need to **create a `PostFlopGame` instance**, | ||
**allocate global storage and `PostFlopNode`s**, and **initialize the | ||
`PostFlopNode` child/parent relationship**. This is done in several steps. | ||
|
||
We begin by creating a `PostFlopGame` instance. | ||
|
||
```rust | ||
let mut game = PostFlopGame::with_config(card_config, action_tree).unwrap(); | ||
``` | ||
|
||
A `PostFlopGame` requires an | ||
`ActionTree` which describes all possible actions and lines (no runout | ||
information), and a `CardConfig`, which describes player ranges and | ||
flop/turn/river data. | ||
|
||
Once we have created a `PostFlopGame` instance we need to allocate the following | ||
memory and initialize its values: | ||
|
||
+ `game.node_arena` | ||
+ `game.storage1` | ||
+ `game.storage2` | ||
+ `game.storage_ip` | ||
+ `game.storage_chance` | ||
|
||
These fields are not allocated/initialized at the same time: | ||
|
||
+ `game.node_arena` is allocated and initialized via `with_config()` (i.e., when | ||
we created our `PostFlopGame`), | ||
+ other storage is allocated via `game.allocate_memory()`. | ||
|
||
#### Allocating and Initializing `node_arena` | ||
|
||
We constructed a `PostFlopGame` by calling | ||
`PostFlopGame::with_config(card_config, action_tree)`, which under the hood | ||
actually calls: | ||
|
||
```rust | ||
let mut game = Self::new(); | ||
game.update_config(card_config, action_tree)?; | ||
``` | ||
|
||
`PostFlopGame::update_config` sets up configuration data, sanity checks things | ||
are correct, and then calls `self.init_root()`. | ||
|
||
`init_root` is responsible for: | ||
|
||
1. Counting number of `PostFlopNode`s to be allocated (`self.nodes_per_street`), | ||
broken up by flop, turn, and river | ||
2. Allocating `PostFlopNode`s in the `node_arena` field | ||
3. Clearing storage: `self.clear_storage()` sets each storage item to a new | ||
`Vec` | ||
4. Invoking `build_tree_recursive` which initializes each node's child/parent | ||
relationship via `child_offset` (through calls to `push_actions` and | ||
`push_chances`). | ||
|
||
Each `PostFlopNode` points to node-specific data (e.g., strategies and | ||
cfregrets) that is located inside of `PostFlopGame.storage*` fields (which is | ||
currently unallocated) via similarly named fields `PostFlopNode.storage*`. | ||
|
||
Additionally, each node points to the children offset with `children_offset`, | ||
which records where in `node_arena` relative to the current node that node's | ||
children begin. We allocate this memory via: | ||
|
||
```rust | ||
game.allocate_memory(false); // pass `true` to use compressed memory | ||
``` | ||
|
||
This allocates the following memory: | ||
|
||
+ `self.storage1` | ||
+ `self.storage2` | ||
+ `self.storage3` | ||
+ `self.storage_chance` | ||
|
||
Next, `allocate_memory()` calls `allocate_memory_nodes(&mut self)`, which | ||
iterates through each node in `node_arena` and sets storage pointers. | ||
|
||
After `allocate_memory` returns we still need to set `child_offset`s. | ||
|
||
### Storage | ||
|
||
There are several fields marked as `// global storage` in `game::mod::PostFlopGame`: | ||
|
||
```rust | ||
// global storage | ||
// `storage*` are used as a global storage and are referenced by `PostFlopNode::storage*`. | ||
// Methods like `PostFlopNode::strategy` define how the storage is used. | ||
node_arena: Vec<MutexLike<PostFlopNode>>, | ||
storage1: Vec<u8>, | ||
storage2: Vec<u8>, | ||
storage_ip: Vec<u8>, | ||
storage_chance: Vec<u8>, | ||
locking_strategy: BTreeMap<usize, Vec<f32>>, | ||
``` | ||
|
||
These are referenced from `PostFlopNode`: | ||
|
||
```rust | ||
storage1: *mut u8, // strategy | ||
storage2: *mut u8, // regrets or cfvalues | ||
storage3: *mut u8, // IP cfvalues | ||
``` | ||
|
||
+ `storage1` seems to store the strategy | ||
+ `storage2` seems to store regrets/cfvalues, and | ||
+ `storage3` stores IP's cf values (does that make `storage2` store OOP's cfvalues?) | ||
|
||
Storage is a byte vector `Vec<u8>`, and these store floating point values. | ||
|
||
> [!IMPORTANT] | ||
> Why are these stored as `Vec<u8>`s? Is this for swapping between | ||
> `f16` and `f32`s? | ||
Some storage is allocated in `game::base::allocate_memory`: | ||
|
||
```rust | ||
let storage_bytes = (num_bytes * self.num_storage) as usize; | ||
let storage_ip_bytes = (num_bytes * self.num_storage_ip) as usize; | ||
let storage_chance_bytes = (num_bytes * self.num_storage_chance) as usize; | ||
|
||
self.storage1 = vec![0; storage_bytes]; | ||
self.storage2 = vec![0; storage_bytes]; | ||
self.storage_ip = vec![0; storage_ip_bytes]; | ||
self.storage_chance = vec![0; storage_chance_bytes]; | ||
``` | ||
|
||
`node_arena` is allocated in `game::base::init_root()`: | ||
|
||
```rust | ||
let num_nodes = self.count_nodes_per_street(); | ||
let total_num_nodes = num_nodes[0] + num_nodes[1] + num_nodes[2]; | ||
|
||
if total_num_nodes > u32::MAX as u64 | ||
|| mem::size_of::<PostFlopNode>() as u64 * total_num_nodes > isize::MAX as u64 | ||
{ | ||
return Err("Too many nodes".to_string()); | ||
} | ||
|
||
self.num_nodes = num_nodes; | ||
self.node_arena = (0..total_num_nodes) | ||
.map(|_| MutexLike::new(PostFlopNode::default())) | ||
.collect::<Vec<_>>(); | ||
self.clear_storage(); | ||
``` | ||
|
||
`locking_strategy` maps node indexes (`PostFlopGame::node_index`) to a locked | ||
strategy. `locking_strategy` is initialized to an empty `BTreeMap<usize, | ||
Vec<f32>>` by deriving Default. It is inserted into via | ||
`PostFlopGame::lock_current_strategy` | ||
|
||
### Serialization/Deserialization | ||
|
||
Serialization relies on the `bincode` library's `Encode` and `Decode`. We can set | ||
the `target_storage_mode` to allow for a non-full save. For instance, | ||
|
||
```rust | ||
game.set_target_storage_mode(BoardState::Turn); | ||
``` | ||
|
||
will ensure that when `game` is encoded, it will only save Flop and Turn data. | ||
When a serialized tree is deserialized, if it is a partial save (e.g., a Turn | ||
save) you will not be able to navigate to unsaved streets. | ||
|
||
Several things break when we deserialize a partial save: | ||
|
||
+ `node_arena` is only partially populated | ||
+ `node.children()` points to raw data when `node` points to an street that is | ||
not serialized (e.g., a chance node before the river for a Turn save). | ||
|
||
### Allocating `node_arena` | ||
|
||
We want to first allocate nodes for `node_arena`, and then run some form of | ||
`build_tree_recursive`. This assumes that `node_arena` is already allocated, and | ||
recursively visits children of nodes and modifies them to | ||
|
||
### Data Coupling/Relations/Invariants | ||
|
||
+ A node is locked IFF it is contained in the game's locking_strategy | ||
+ `PostFlopGame.node_arena` is pointed to by `PostFlopNode.children_offset`. For | ||
instance, this is the basic definition of the `PostFlopNode.children()` | ||
function: | ||
|
||
```rust | ||
slice::from_raw_parts( | ||
self_ptr.add(self.children_offset as usize), | ||
self.num_children as usize, | ||
) | ||
``` | ||
|
||
We get a pointer to `self` and add children offset. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
use postflop_solver::*; | ||
|
||
fn main() { | ||
// ranges of OOP and IP in string format | ||
// see the documentation of `Range` for more details about the format | ||
let oop_range = "66+"; | ||
let ip_range = "66+"; | ||
|
||
let card_config = CardConfig { | ||
range: [oop_range.parse().unwrap(), ip_range.parse().unwrap()], | ||
flop: flop_from_str("Td9d6h").unwrap(), | ||
turn: NOT_DEALT, | ||
river: NOT_DEALT, | ||
}; | ||
|
||
// bet sizes -> 60% of the pot, geometric size, and all-in | ||
// raise sizes -> 2.5x of the previous bet | ||
// see the documentation of `BetSizeOptions` for more details | ||
let bet_sizes = BetSizeOptions::try_from(("100%", "100%")).unwrap(); | ||
|
||
let tree_config = TreeConfig { | ||
initial_state: BoardState::Flop, // must match `card_config` | ||
starting_pot: 200, | ||
effective_stack: 200, | ||
rake_rate: 0.0, | ||
rake_cap: 0.0, | ||
flop_bet_sizes: [bet_sizes.clone(), bet_sizes.clone()], // [OOP, IP] | ||
turn_bet_sizes: [bet_sizes.clone(), bet_sizes.clone()], | ||
river_bet_sizes: [bet_sizes.clone(), bet_sizes], | ||
turn_donk_sizes: None, // use default bet sizes | ||
river_donk_sizes: Some(DonkSizeOptions::try_from("100%").unwrap()), | ||
add_allin_threshold: 1.5, // add all-in if (maximum bet size) <= 1.5x pot | ||
force_allin_threshold: 0.15, // force all-in if (SPR after the opponent's call) <= 0.15 | ||
merging_threshold: 0.1, | ||
}; | ||
|
||
// build the game tree | ||
// `ActionTree` can be edited manually after construction | ||
let action_tree = ActionTree::new(tree_config).unwrap(); | ||
let mut game = PostFlopGame::with_config(card_config, action_tree).unwrap(); | ||
|
||
// allocate memory without compression (use 32-bit float) | ||
game.allocate_memory(false); | ||
|
||
// solve the game | ||
let max_num_iterations = 20; | ||
let target_exploitability = game.tree_config().starting_pot as f32 * 0.100; // 10.0% of the pot | ||
let exploitability = solve(&mut game, max_num_iterations, target_exploitability, true); | ||
println!("Exploitability: {:.2}", exploitability); | ||
|
||
// get equity and EV of a specific hand | ||
game.cache_normalized_weights(); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.