All notable changes to this module will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Replaced all
pgx.Conn
withpgxpool.Conn
to use postgres connection pooling - Consolidate and simplify
Release
andClose
andresetContext
into a single function - Introduced a
ConnectTimeout
to the postgres connection - Remove the unnecessary
getTx
helper - Maintain a pool of connections to the postgres DB rather than creating a new one each time
- Improve readability of
ReleaseWriteContext
- Updated to use
logger
where we were still usinglog
for the message "Storing block in block store"
- Update state hash test after modifying genesis (updated port numbers)
- Renamed package names and parameters to reflect changes in the rest of the codebase
- Removed the
StakeStatus
constants to use the global proto enum types - Removed the whole concept of
maxRelays
as we're introducing a new rate limiting approach - Added documentation to make certain things clearer
- Renamed
GenericParam
toServiceURL
- Removed
txsOrderInBlockHashDescending
since we don't need a hash specific to txs - it's part of the state hash
- Update logger value references with pointers
- Remove unnecessary (and problematic) logger calls
- Rename ServiceNode Actor Type Name to Servicer
- Module now embeds
base_modules.IntegratableModule
for DRYness
- Add a few
nolint
comments to fix the code on main
- Remove
IUnstakingActor
and useUnstakingActor
directly; guideline for removing future unnecessary types (e.g. TxResult) - Typo in
GetMinimumBlockHeightQuery
- Reduce unnecessary
string
<->[]byte
conversion in a few places - Fix bug in
updateUnstakedHeightIfPausedBefore
that was unstaking all actors
- Added mock generation to the
kvstore/kvstore.go
.
- Minor documentation cleanup
- Address legacy linter errors from
golangci-lint
- Changed log lines to utilize new logger module
- Use hash of serialised protobufs for keys in
updateParamsTree()
andupdateFlagsTree()
- Fix unit tests -
TestGetAppPauseHeightIfExists
,TestGetAppOutputAddress
,TestGetFishermanStatus
,TestGetFishermanPauseHeightIfExists
,TestGetFishermanOutputAddress
,TestPersistenceContextParallelReadWrite
,TestGetServicerPauseHeightIfExists
,TestGetServicerOutputAddress
,fuzzSingleProtocolActor
,TestGetValidatorPauseHeightIfExists
, andTestGetValidatorOutputAddress
for misplaced expected and actual values inrequire.Equal
.
- Add logic for
updateParamsTree()
andupdateFlagsTree()
functions when updating merkle root hash
- Added
debug.FreeOSMemory()
onResetToGenesis
to free-up memory and stabilizeLocalNet
.
- Consolidate common behaviour of
Pool
andAccount
functions into a shared interfaceProtocolAccountSchema
- Create
account_shared_sql.go
andtypes/account_shared_sql.go
and renameshared_sql.go
andtype/shared_sql.go
toactor_shared_sql.go
andtypes/actor_shared_sql.go
seperating shared sql logic
- Update the persistence module README, focusing on
pgadmin
and Makefile helpers
- Remove
Block
proto definition to consolidate undershared/core/types
- Add
max_conns_count
,min_conns_count
,max_conn_lifetime
,max_conn_idle_time
andhealth_check_period
toPersistenceConfig
. - Update
connectToDatabase
function indb.go
to connect viapgxpool
to postgres database and acceptPersistenceConfig
interface as input. - Update
github.com/jackc/pgx/v4
->github.com/jackc/pgx/v5
.
- Add
init()
function togov.go
to build a map of parameter names and their types - Deprecated
GetBlocksPerSession()
andGetServicersPerSessionAt()
in favour of the more general parameter getter functionGetParameter()
- Update unit tests replacing
GetIntParam()
andGetStringParam()
calls withGetParameter()
- Minor logging improvements
- Updated module constructor to accept a
bus
and not aruntimeMgr
anymore - Registering module with the
bus
viaRegisterModule
method - Updated tests and mocks accordingly
- Renamed
InitParams
toInitGenesisParams
- Added missing
ActorType
inGetAllXXXX()
functions - Updated to new
PoolNames
enums - Using
Enum.FriendlyName()
instead ofEnum.String()
forPoolNames
enums (backward compatibility + flexibility) - Updated
InitParams
so that Params can be initialized from aGenesisState
and not just hardcoded - Refactored default values sourcing (test_artifacts for tests)
- Updated tests
- Consolidated
persistence/docs/CHANGELOG
andpersistence/CHANGELOG.md
intopersistence/docs/CHANGELOG
- Updated to use centralized config and genesis
- Updated to use
Account
struct now undercoreTypes
- Tended for the TODO "// TODO (Andrew) genericize the genesis population logic for actors #149" in
persistence/genesis.go
- Updated tests to use the new config and genesis handling
- Updated statetest hashes to reflect updated genesis state
- Remove
SetProposalBlock
and local vars to keep proposal state - Add
proposerAddr
to theCommit
function - Move the
PostgresContext
struct tocontext.db
- Moved Actor related getters from
genesis.go
toactor.go
- Added
GetAllStakedActors()
that returns all Actors
- Changed the scope of
TransactionExists
from thePostgresContext
to thePersistenceModule
Core StateHash changes
- Introduced & defined for
block_persistence.proto
- A persistence specific protobuf for the Block stored in the BlockStore
- On
Commit
, prepare and store a persistence block in the KV Store, SQL Store - Replace
IndexTransactions
(plural) toIndexTransaction
(singular) - Maintaining a list of StateTrees using Celestia’s SMT and badger as the KV store to compute the state hash
- Implemented
ComputeStateHash
to update the global state based on:- Validators
- Applications
- Servicers
- Fisherman
- Accounts
- Pools
- Transactions
- Added a placeholder for
params
andflags
- Added a benchmarking and a determinism test suite to validate this
Supporting StateHash changes
- Implemented
GetAccountsUpdated
,GetPoolsUpdated
andGetActorsUpdated
functions - Removed
GetPrevAppHash
andindexTransactions
functions - Removed
blockProtoBytes
andtxResults
from the local state and addedquorumCert
- Consolidate all
resetContext
related operations into a single function - Implemented
ReleaseWriteContext
- Implemented ability to
ClearAllState
andResetToGenesis
for debugging & testing purposes - Added unit tests for all of the supporting SQL functions implemented
- Some improvements in unit test preparation & cleanup (limited to this PR's functionality)
KVStore changes
- Renamed
Put
toSet
- Embedded
smt.MapStore
in the interface containingGet
,Set
andDelete
- Implemented
Delete
- Modified
GetAll
to return bothkeys
andvalues
- Turned off badger logging options since it’s noisy
- Rename
GetBlockHash
toGetBlockHashAtHeight
- Reduce visibility scope of
IndexTransactions
toindexTransactions
- Remove
quorumCertificate
from the local context state - Remove
LatestQC
andSetLatestQC
- Remove
Latest
prefix from several functions including related to setting context of the proposal block - Added
ReleaseWriteContext
placeholder - Replaced
ResetContext
withRelease
- Changed the following exported functions to lowercase non-exported functions
- [./pocket/persistence/]
- GetActor
- GetActorFromRow
- GetChainsForActor
- SetActorStakeAmount
- GetActorStakeAmount
- GetCtxAndTx
- GetCtx
- SetValidatorStakedTokens
- GetValidatorStakedTokens
- [./pocket/persistence/types]
- ProtocolActorTableSchema
- ProtocolActorChainsTableSchema
- SelectChains
- ReadyToUnstake
- InsertChains
- UpdateUnstakingHeight
- UpdateStakeAmount
- UpdatePausedHeight
- UpdateUnstakedHeightIfPausedBefore
- AccToAccInterface
- TestInsertParams
- AccountOrPoolSchema
- InsertAcc
- SelectBalance
- [./pocket/persistence/test]
- GetGenericActor
- NewTestGenericActor
- Ported over storing blocks and block components to the Persistence module from Consensus and Utility modules
- Encapsulated
TxIndexer
logic to the persistence context only
- Fixed
ToPersistenceActors()
by filling all structure fields - Deprecated
BaseActor
->Actor
- Changed default actor type to
ActorType_Undefined
#235 Config and genesis handling
- Updated to use
RuntimeMgr
- Made
PersistenceModule
struct unexported - Updated tests and mocks
- Removed some cross-module dependencies
- Added
TxIndexer
sub-package (previously in Utility Module) - Added
TxIndexer
to bothPersistenceModule
andPersistenceContext
- Implemented
TransactionExists
andStoreTransaction
- Don't ignore the exit code of
m.Run()
in the unit tests - Fixed several broken unit tests related to type casting
- Removed no-op
DeleteActor
code - Consolidated
CHANGELOG
s into one underpersistence/docs
- Consolidated
README
s into one underpersistence/docs
- Deprecated
persMod.ResetContext()
for ->persRWContext.ResetContext()
for more appropriate encapsulation - Added ticks to CHANGELOG.md
- Removed reference to Utility Mod's
BigIntToString()
and used internalBigIntToString()
- Consolidated
PostgresContext
andPostgresDb
into a single structure
Encapsulate structures previously in shared #163
- Renamed schema -> types
- Added genesis, config, and unstaking proto files from shared
- Ensured proto structures implement shared interfaces
- Populate
PersistenceGenesisState
uses shared interfaces in order to acceptMockPersistenceGenesisState
- ^ Same applies for
PersistenceConfig
- Bumped cleanup TODOs to #149 due to scope size of #163
Main persistence module changes:
- Split
ConnectAndInitializeDatabase
intoconnectToDatabase
andinitializeDatabase
- This enables creating multiple contexts in parallel without re-initializing the DB connection
- Fix the SQL query used in
SelectActors
,SelectAccounts
&SelectPools
- Add a generalized unit test for all actors
- Remove
NewPersistenceModule
and an injectedConfig
+Create
- This improves isolation a a “injection-like” paradigm for unit testing
- Change
SetupPostgresDocker
toSetupPostgresDockerPersistenceMod
- This enables more “functional” like testing by returning a persistence module and avoiding global testing variables
- Only return once a connection to the DB has been initialized reducing the likelihood of test race conditions
- Implemented
NewReadContext
with a proper read-only context - Add
ResetContext
to the persistence module andClose
to the read context
Secondary persistence module changes
- Improve return values in
Commit
andRelease
(return error, add logging, etc…) - Add
pgx.Conn
pointer toPostgresDB
s/db/conn/g
ands/conn/tx/g
in some (not all) places where appropriate- Make some exported variables / functions unexported for readability & access purposes
- Add a few helpers for persistence related unit testing
- Added unit tests and TODOs for handling multiple read/write contexts
Deprecate PrePersistence
- Fix for bytes parameters
- Accounts / pools default to 0
- Pre-added accounts to genesis file
- Separated out Persistence Read Context from Persistence Write Context
- Added various TODO's in order to code-complete a working persistence module
- Added genesis level functions to GetAllActors() and GetAllAccounts/Pools() for testing
- Added PopulateGenesisState function to persistence module
- Fixed the stake status iota issue
- Discovered and documented (with TODO) double setting parameters issue
- Attached to the Utility Module and using in
make compose_and_watch
Pocket Persistence 1st Iteration (pokt-network#73)
- Base persistence module implementation for the following actors:
Account
,Pool
,Validator
,Fisherman
,Servicer
,Application
- Generalization of common protocol actor behvaiours via the
ProtocolActor
andBaseActor
interface and implementation - A PostgreSQL based implementation of the persistence middleware including:
- SQL query implementation for each actor
- SQL schema definition for each actor
- SQL execution for common actor behaviours
- Golang interface implementation of the Persistence module
- Update to the Persistence module interface to enable historical height queries
- Library / infrastructure for persistence unit fuzz testing
- Tests triggered via
make test_persistence