Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add tx status in mempool #1287

Merged
merged 49 commits into from
Jun 18, 2024
Merged
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
473937e
feat: add tx status in mempool
ninabarbakadze Apr 5, 2024
33355a0
refactor: LRUTxCache to take tx key instead of tx
ninabarbakadze Apr 18, 2024
d4a86e3
feat: add the same methods to other mempools to build (wip)
ninabarbakadze Apr 19, 2024
cd28593
chore: fix go sum entry error
ninabarbakadze May 17, 2024
aff84b8
(wip): remove rejected, evicted txs works
ninabarbakadze Jun 5, 2024
4039a56
refactor: tests and functions to make it cleaner
ninabarbakadze Jun 6, 2024
51f2e96
Merge branch 'main' into nina/current-tx-status
ninabarbakadze Jun 6, 2024
6fb9af9
fix: build
ninabarbakadze Jun 6, 2024
2f38c2b
test: more tests
ninabarbakadze Jun 6, 2024
edbcdae
style: lint
ninabarbakadze Jun 6, 2024
652f814
style: cleanup
ninabarbakadze Jun 6, 2024
2cede15
style: lint
ninabarbakadze Jun 6, 2024
125ef6f
style: gofumped
ninabarbakadze Jun 6, 2024
37c8cd1
style: goimports
ninabarbakadze Jun 6, 2024
90f482c
Update mempool/cache.go
ninabarbakadze Jun 11, 2024
5cdd6e0
Update mempool/cache.go
ninabarbakadze Jun 11, 2024
94a4d3d
Update mempool/cache.go
ninabarbakadze Jun 11, 2024
26bbd60
Update mempool/cat/pool.go
ninabarbakadze Jun 11, 2024
7708045
Update mempool/mempool.go
ninabarbakadze Jun 11, 2024
4b8918d
Update mempool/mock/mempool.go
ninabarbakadze Jun 11, 2024
2a07da9
Update mempool/v0/clist_mempool.go
ninabarbakadze Jun 11, 2024
e8e634b
Update mempool/v0/clist_mempool.go
ninabarbakadze Jun 11, 2024
e91c8f1
Update rpc/core/tx_status.go
ninabarbakadze Jun 11, 2024
9bcd19b
Update mempool/v1/mempool.go
ninabarbakadze Jun 11, 2024
c370030
refactor: address review comments
ninabarbakadze Jun 11, 2024
1cbb37d
test: add more consistent comments
ninabarbakadze Jun 11, 2024
a22978d
Update pool.go
ninabarbakadze Jun 11, 2024
33e8bf3
Update mempool.go
ninabarbakadze Jun 11, 2024
709e6d3
Update mempool/v1/mempool.go
ninabarbakadze Jun 12, 2024
39f1deb
Update mempool/v1/mempool.go
ninabarbakadze Jun 12, 2024
2384c48
Merge branch 'main' into nina/current-tx-status
ninabarbakadze Jun 12, 2024
58929bb
Update mempool/cat/pool_test.go
ninabarbakadze Jun 12, 2024
840b61d
Update mempool/cat/pool.go
ninabarbakadze Jun 12, 2024
d21e084
refacotor: address review comments
ninabarbakadze Jun 13, 2024
486888e
Update mempool.go
ninabarbakadze Jun 13, 2024
8a5bbae
refactor: revert mempool cache changes
ninabarbakadze Jun 13, 2024
e4c9839
style: lint
ninabarbakadze Jun 13, 2024
5ae87e9
refactor: comments
ninabarbakadze Jun 13, 2024
6ac1205
refactor: revert accidentally removed comment
ninabarbakadze Jun 13, 2024
4e09a6c
Update mempool/v1/mempool.go
ninabarbakadze Jun 13, 2024
cc8c178
Update mempool/mempool.go
ninabarbakadze Jun 13, 2024
7437fd9
Update mempool/mempool.go
ninabarbakadze Jun 13, 2024
c6f3e9a
test: update test
ninabarbakadze Jun 13, 2024
bc74ef3
refactor: no longer pass the cache in purgeExpiredTxs and reduce evic…
ninabarbakadze Jun 17, 2024
64eb498
feat: add txstatus in mempool and test it
ninabarbakadze Jun 17, 2024
bbcb66a
style: lint
ninabarbakadze Jun 17, 2024
a75c6d5
lint: remove lint error
ninabarbakadze Jun 17, 2024
14d09b0
refactor: rename IsTxEvicted to WasRecentlyEvicted
ninabarbakadze Jun 17, 2024
8e3c51d
Update mempool/cat/store.go
ninabarbakadze Jun 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions consensus/replay_stubs.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,10 @@ func (emptyMempool) TxsBytes() int64 { return 0 }
func (emptyMempool) TxsFront() *clist.CElement { return nil }
func (emptyMempool) TxsWaitChan() <-chan struct{} { return nil }

func (emptyMempool) InitWAL() error { return nil }
func (emptyMempool) CloseWAL() {}
func (emptyMempool) InitWAL() error { return nil }
func (emptyMempool) CloseWAL() {}
func (emptyMempool) GetTxByKey(types.TxKey) (types.Tx, bool) { return nil, false }
func (emptyMempool) IsTxEvicted(types.TxKey) bool { return false }

//-----------------------------------------------------------------------------
// mockProxyApp uses ABCIResponses to give the right results.
Expand Down
17 changes: 9 additions & 8 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ require (
github.com/go-logfmt/logfmt v0.5.1
github.com/gofrs/uuid v4.3.0+incompatible
github.com/gogo/protobuf v1.3.2
github.com/golang/mock v1.4.4
github.com/golang/protobuf v1.5.3
github.com/golangci/golangci-lint v1.50.1
github.com/google/orderedcode v0.0.1
Expand Down Expand Up @@ -49,8 +50,8 @@ require (
go.opentelemetry.io/otel v1.21.0
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.18.0
go.opentelemetry.io/otel/sdk v1.21.0
golang.org/x/crypto v0.21.0
golang.org/x/net v0.23.0
golang.org/x/crypto v0.24.0
golang.org/x/net v0.26.0
gonum.org/v1/gonum v0.8.2
google.golang.org/grpc v1.59.0
google.golang.org/protobuf v1.31.0
Expand Down Expand Up @@ -275,12 +276,12 @@ require (
go.uber.org/zap v1.23.0 // indirect
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e // indirect
golang.org/x/exp/typeparams v0.0.0-20220827204233-334a2380cb91 // indirect
golang.org/x/mod v0.12.0 // indirect
golang.org/x/sync v0.3.0 // indirect
golang.org/x/sys v0.18.0 // indirect
golang.org/x/term v0.18.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/tools v0.13.0 // indirect
golang.org/x/mod v0.18.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.21.0 // indirect
golang.org/x/term v0.21.0 // indirect
golang.org/x/text v0.16.0 // indirect
golang.org/x/tools v0.22.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
Expand Down
33 changes: 17 additions & 16 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,7 @@ github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFU
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.4 h1:l75CXGRSwbaYNpl/Z2X1XIIAMSCquvXgpVZDhwEIJsc=
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
Expand Down Expand Up @@ -989,8 +990,8 @@ golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5y
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.3.1-0.20221117191849-2c476679df9a/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
golang.org/x/crypto v0.24.0 h1:mnl8DM0o513X8fdIkmyFE/5hTYxbwYOjDS/+rK6qpRI=
golang.org/x/crypto v0.24.0/go.mod h1:Z1PMYSOR5nyMcyAVAIQSKCDwalqy85Aqn1x3Ws4L5DM=
golang.org/x/exp v0.0.0-20180321215751-8460e604b9de/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20180807140117-3d87b88a115f/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
Expand Down Expand Up @@ -1038,8 +1039,8 @@ golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0 h1:rmsUpXtvNzj340zd98LZ4KntptpfRHwpFOHG188oHXc=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.18.0 h1:5+9lSbEzPSdWkH32vYPBwEpX8KwDbM52Ud9xBUvNlb0=
golang.org/x/mod v0.18.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20180719180050-a680a1efc54d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
Expand Down Expand Up @@ -1089,8 +1090,8 @@ golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/net v0.23.0 h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
golang.org/x/net v0.26.0 h1:soB7SVo0PWrY4vPW/+ay0jKDNScG2X9wFeYlXIvJsOQ=
golang.org/x/net v0.26.0/go.mod h1:5YKkiSynbBIh3p6iOc/vibscux0x38BZDkn8sCUPxHE=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
Expand All @@ -1117,8 +1118,8 @@ golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
Expand Down Expand Up @@ -1200,15 +1201,15 @@ golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.21.0 h1:rF+pYz3DAGSQAxAu1CbC7catZg4ebC4UIeIhKxBZvws=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
golang.org/x/term v0.18.0 h1:FcHjZXDMxI8mM3nwhX9HlKop4C0YQvCVCdwYl2wOtE8=
golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58=
golang.org/x/term v0.21.0 h1:WVXCp+/EBEHOj53Rvu+7KiT/iElMrO8ACK16SMZ3jaA=
golang.org/x/term v0.21.0/go.mod h1:ooXLefLobQVslOqselCNF4SxFAaoS6KujMbsGzSDmX0=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
Expand All @@ -1220,8 +1221,8 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
Expand Down Expand Up @@ -1312,8 +1313,8 @@ golang.org/x/tools v0.1.10/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E
golang.org/x/tools v0.1.11/go.mod h1:SgwaegtQh8clINPpECJMqnxLv9I09HLqnW3RMqW0CA4=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0 h1:Iey4qkscZuv0VvIt8E0neZjtPVQFSc870HQ448QgEmQ=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.22.0 h1:gqSGLZqv+AI9lIQzniJ0nZDRG5GBPsSi+DRNHWNz6yA=
golang.org/x/tools v0.22.0/go.mod h1:aCwcsjqvq7Yqt6TNyX7QMU2enbQ/Gt0bo6krSeEri+c=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
Expand Down
20 changes: 16 additions & 4 deletions mempool/cache.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ type TxCache interface {
// Has reports whether tx is present in the cache. Checking for presence is
// not treated as an access of the value.
ninabarbakadze marked this conversation as resolved.
Show resolved Hide resolved
Has(tx types.Tx) bool

// HasKey reports whether the given key is present in the cache.
HasKey(key types.TxKey) bool
}

var _ TxCache = (*LRUTxCache)(nil)
Expand Down Expand Up @@ -113,12 +116,21 @@ func (c *LRUTxCache) Has(tx types.Tx) bool {
return ok
}

func (c *LRUTxCache) HasKey(key types.TxKey) bool {
c.mtx.Lock()
defer c.mtx.Unlock()

_, ok := c.cacheMap[key]
return ok
}

// NopTxCache defines a no-op raw transaction cache.
type NopTxCache struct{}

var _ TxCache = (*NopTxCache)(nil)

func (NopTxCache) Reset() {}
func (NopTxCache) Push(types.Tx) bool { return true }
func (NopTxCache) Remove(types.Tx) {}
func (NopTxCache) Has(types.Tx) bool { return false }
func (NopTxCache) Reset() {}
func (NopTxCache) Push(types.Tx) bool { return true }
func (NopTxCache) Remove(types.Tx) {}
func (NopTxCache) Has(types.Tx) bool { return false }
func (NopTxCache) HasKey(types.TxKey) bool { return false }
26 changes: 22 additions & 4 deletions mempool/cat/pool.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ type TxPool struct {

// Thread-safe cache of rejected transactions for quick look-up
rejectedTxCache *LRUTxCache
// Thread-safe cache of evicted transactions for quick look-up
evictedTxCache *LRUTxCache
// Thread-safe list of transactions peers have seen that we have not yet seen
seenByPeersSet *SeenTxSet

Expand Down Expand Up @@ -92,6 +94,7 @@ func NewTxPool(
proxyAppConn: proxyAppConn,
metrics: mempool.NopMetrics(),
rejectedTxCache: NewLRUTxCache(cfg.CacheSize),
evictedTxCache: NewLRUTxCache(cfg.CacheSize),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we perhaps make this 1/5 the size of the rejectedTxCache. The default in the config is currently 10k, so I think keeping the last 2k of evicted transactions is sufficient

seenByPeersSet: NewSeenTxSet(),
height: height,
preCheckFn: func(_ types.Tx) error { return nil },
Expand Down Expand Up @@ -171,16 +174,28 @@ func (txmp *TxPool) Has(txKey types.TxKey) bool {
return txmp.store.has(txKey)
}

// Get retrieves a transaction based on the key. It returns a bool
// if the transaction exists or not
// Get retrieves a transaction based on the key.
// Deprecated: use GetTxByKey instead.
func (txmp *TxPool) Get(txKey types.TxKey) (types.Tx, bool) {
return txmp.GetTxByKey(txKey)
}

// GetTxByKey retrieves a transaction based on the key. It returns a bool
// indicating whether transaction was found in the cache.
func (txmp *TxPool) GetTxByKey(txKey types.TxKey) (types.Tx, bool) {
ninabarbakadze marked this conversation as resolved.
Show resolved Hide resolved
wtx := txmp.store.get(txKey)
if wtx != nil {
return wtx.tx, true
}
return types.Tx{}, false
}

// IsTxEvicted returns a bool indicating whether the transaction with
// the specified key was recently evicted and is currently within the cache.
func (txmp *TxPool) IsTxEvicted(txKey types.TxKey) bool {
return txmp.evictedTxCache.Has(txKey)
}

// IsRejectedTx returns true if the transaction was recently rejected and is
// currently within the cache
func (txmp *TxPool) IsRejectedTx(txKey types.TxKey) bool {
Expand All @@ -197,7 +212,7 @@ func (txmp *TxPool) CheckToPurgeExpiredTxs() {
expirationAge := time.Now().Add(-txmp.config.TTLDuration)
// a height of 0 means no transactions will be removed because of height
// (in other words, no transaction has a height less than 0)
numExpired := txmp.store.purgeExpiredTxs(0, expirationAge)
numExpired := txmp.store.purgeExpiredTxs(0, expirationAge, txmp.evictedTxCache)
txmp.metrics.EvictedTxs.Add(float64(numExpired))
txmp.lastPurgeTime = time.Now()
}
Expand Down Expand Up @@ -373,6 +388,7 @@ func (txmp *TxPool) Flush() {
txmp.store.reset()
txmp.seenByPeersSet.Reset()
txmp.rejectedTxCache.Reset()
txmp.evictedTxCache.Reset()
txmp.metrics.EvictedTxs.Add(float64(size))
txmp.broadcastMtx.Lock()
defer txmp.broadcastMtx.Unlock()
Expand Down Expand Up @@ -537,6 +553,7 @@ func (txmp *TxPool) addNewTransaction(wtx *wrappedTx, checkTxRes *abci.ResponseC
// drop the new one.
if len(victims) == 0 || victimBytes < wtx.size() {
txmp.metrics.EvictedTxs.Add(1)
txmp.evictedTxCache.Push(wtx.key)
checkTxRes.MempoolError = fmt.Sprintf("rejected valid incoming transaction; mempool is full (%X)",
wtx.key)
return fmt.Errorf("rejected valid incoming transaction; mempool is full (%X). Size: (%d:%d)",
Expand Down Expand Up @@ -591,6 +608,7 @@ func (txmp *TxPool) addNewTransaction(wtx *wrappedTx, checkTxRes *abci.ResponseC

func (txmp *TxPool) evictTx(wtx *wrappedTx) {
txmp.store.remove(wtx.key)
txmp.evictedTxCache.Push(wtx.key)
txmp.metrics.EvictedTxs.Add(1)
txmp.logger.Debug(
"evicted valid existing transaction; mempool full",
Expand Down Expand Up @@ -720,7 +738,7 @@ func (txmp *TxPool) purgeExpiredTxs(blockHeight int64) {
expirationAge = time.Time{}
}

numExpired := txmp.store.purgeExpiredTxs(expirationHeight, expirationAge)
numExpired := txmp.store.purgeExpiredTxs(expirationHeight, expirationAge, txmp.evictedTxCache)
txmp.metrics.EvictedTxs.Add(float64(numExpired))

// purge old evicted and seen transactions
Expand Down
15 changes: 14 additions & 1 deletion mempool/cat/pool_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,7 @@ func TestTxPool_Eviction(t *testing.T) {
mustCheckTx(t, txmp, "key1=0000=25")
require.True(t, txExists("key1=0000=25"))
require.False(t, txExists(bigTx))
require.True(t, txmp.IsTxEvicted(types.Tx(bigTx).Key()))
require.Equal(t, int64(len("key1=0000=25")), txmp.SizeBytes())

// Now fill up the rest of the slots with other transactions.
Expand All @@ -257,23 +258,27 @@ func TestTxPool_Eviction(t *testing.T) {
require.Error(t, err)
require.Contains(t, err.Error(), "mempool is full")
require.False(t, txExists("key6=0005=1"))
require.True(t, txmp.IsTxEvicted(types.Tx("key6=0005=1").Key()))

// A new transaction with higher priority should evict key5, which is the
// newest of the two transactions with lowest priority.
mustCheckTx(t, txmp, "key7=0006=7")
require.True(t, txExists("key7=0006=7")) // new transaction added
require.False(t, txExists("key5=0004=3")) // newest low-priority tx evicted
require.True(t, txExists("key4=0003=3")) // older low-priority tx retained
require.True(t, txmp.IsTxEvicted(types.Tx("key5=0004=3").Key()))
require.True(t, txExists("key4=0003=3")) // older low-priority tx retained

// Another new transaction evicts the other low-priority element.
mustCheckTx(t, txmp, "key8=0007=20")
require.True(t, txExists("key8=0007=20"))
require.False(t, txExists("key4=0003=3"))
require.True(t, txmp.IsTxEvicted(types.Tx("key4=0003=3").Key()))

// Now the lowest-priority tx is 5, so that should be the next to go.
mustCheckTx(t, txmp, "key9=0008=9")
require.True(t, txExists("key9=0008=9"))
require.False(t, txExists("key2=0001=5"))
require.True(t, txmp.IsTxEvicted(types.Tx("key2=0001=5").Key()))

// Add a transaction that requires eviction of multiple lower-priority
// entries, in order to fit the size of the element.
Expand All @@ -282,8 +287,11 @@ func TestTxPool_Eviction(t *testing.T) {
require.True(t, txExists("key8=0007=20"))
require.True(t, txExists("key10=0123456789abcdef=11"))
require.False(t, txExists("key3=0002=10"))
require.True(t, txmp.IsTxEvicted(types.Tx("key3=0002=10").Key()))
require.False(t, txExists("key9=0008=9"))
require.True(t, txmp.IsTxEvicted(types.Tx("key9=0008=9").Key()))
require.False(t, txExists("key7=0006=7"))
require.True(t, txmp.IsTxEvicted(types.Tx("key7=0006=7").Key()))

// Free up some space so we can add back previously evicted txs
err = txmp.Update(1, types.Txs{types.Tx("key10=0123456789abcdef=11")}, []*abci.ResponseDeliverTx{{Code: abci.CodeTypeOK}}, nil, nil)
Expand All @@ -296,6 +304,7 @@ func TestTxPool_Eviction(t *testing.T) {
// space for the previously evicted tx
require.NoError(t, txmp.RemoveTxByKey(types.Tx("key8=0007=20").Key()))
require.False(t, txExists("key8=0007=20"))
require.False(t, txmp.IsTxEvicted(types.Tx("key8=0007=20").Key()))
}

func TestTxPool_Flush(t *testing.T) {
Expand Down Expand Up @@ -567,6 +576,10 @@ func TestTxPool_ExpiredTxs_Timestamp(t *testing.T) {

// All the transactions in the original set should have been purged.
for _, tx := range added1 {
// Check that it was added to the evictedTxCache
evicted := txmp.IsTxEvicted(tx.tx.Key())
require.True(t, evicted)

if txmp.store.has(tx.tx.Key()) {
t.Errorf("Transaction %X should have been purged for TTL", tx.tx.Key())
}
Expand Down
2 changes: 1 addition & 1 deletion mempool/cat/reactor.go
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,7 @@ func (memR *Reactor) ReceiveEnvelope(e p2p.Envelope) {
txKey[:],
schema.Download,
)
tx, has := memR.mempool.Get(txKey)
tx, has := memR.mempool.GetTxByKey(txKey)
if has && !memR.opts.ListenOnly {
peerID := memR.ids.GetIDForPeer(e.Src.ID())
memR.Logger.Debug("sending a tx in response to a want msg", "peer", peerID)
Expand Down
5 changes: 4 additions & 1 deletion mempool/cat/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -142,14 +142,17 @@ func (s *store) getTxsBelowPriority(priority int64) ([]*wrappedTx, int64) {

// purgeExpiredTxs removes all transactions that are older than the given height
// and time. Returns the amount of transactions that were removed
ninabarbakadze marked this conversation as resolved.
Show resolved Hide resolved
func (s *store) purgeExpiredTxs(expirationHeight int64, expirationAge time.Time) int {
func (s *store) purgeExpiredTxs(expirationHeight int64, expirationAge time.Time, evictedTxCache *LRUTxCache) int {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[no change needed] it seems like purge could be renamed to evict here. IMO it's easier to understand what's happening if we use consistent names for consistent things.

In other words, it isn't immediately obvious to me that purging a transaction should add it to the evicted tx cache. However, evicting a transaction seems like it should add the tx to the evicted tx cache.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the cache been passed into the store. It feels leaky. Either the caches are built into the store or this method returns an array of evicted tx keys that the mempool can then plug into the cache. I think I would prefer the latter

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I returns purged txs from the store but i'm afraid that's also api breaking?

s.mtx.Lock()
defer s.mtx.Unlock()
counter := 0
for key, tx := range s.txs {
if tx.height < expirationHeight || tx.timestamp.Before(expirationAge) {
s.bytes -= tx.size()
delete(s.txs, key)
if evictedTxCache != nil {
evictedTxCache.Push(key)
}
counter++
}
}
Expand Down
Loading
Loading