Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

P2P #22

Draft
wants to merge 99 commits into
base: main
Choose a base branch
from
Draft

P2P #22

Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
8ccd71f
theoritcally works, not sure why type issues
gluax Mar 16, 2022
55d3444
piping without it-pipe
gluax Mar 17, 2022
ce284be
add some todos so I don't forget
gluax Mar 17, 2022
00537f2
some cleanup, and saving of known peers
gluax Mar 17, 2022
8f2ad61
logging
gluax Mar 18, 2022
e87c575
clean up saving and loading peers
gluax Mar 18, 2022
b008da5
fix some typing issues
gluax Mar 21, 2022
c61ed73
fix saving of peers
gluax Mar 21, 2022
a23c86b
remove old comments
gluax Mar 23, 2022
18b434f
some slight more cleanup
gluax Mar 23, 2022
1f8657f
changes so far, have a question
gluax Apr 14, 2022
fa91005
aggregation
gluax Apr 19, 2022
ddfc429
merge main, config handling
gluax Apr 19, 2022
0d3d41e
make suggested changes
gluax Apr 20, 2022
cd0f1aa
uses fs promises
gluax Apr 20, 2022
cee0f4f
feat: allow node consensus of chosen median
gluax Apr 25, 2022
1ab178e
feat: Add logging to p2p aggregation utils, add p2p module
gluax Apr 28, 2022
b3dd758
p2p module uses p2p
gluax May 1, 2022
7dfbac0
fix: fix some string bugs in aggrate, and timing of methods called in…
gluax May 1, 2022
3273a78
feat: p2p aggregator use Big.js for better decimal tracking
gluax May 2, 2022
d28ff98
fix: fix equality check of medians with bigjs eq method
gluax May 2, 2022
b8bc9a6
feat: send median to contract
gluax May 9, 2022
ba7eb11
Merge branch 'main' of github.com:fluxprotocol/fpo-node into feature/p2p
gluax May 9, 2022
76a5a3b
feat: started deviation, started fixing some issues with sending/reci…
gluax May 9, 2022
2c5300f
WIP: Easier usage
FranklinWaller May 10, 2022
87c0ae7
WIP: Sending transaction to chain
FranklinWaller May 23, 2022
4986a63
feat(p2p): add protection for double processing
FranklinWaller May 23, 2022
a665f71
fix: generate valid sigs, and correctly grab roundid
gluax Jun 2, 2022
467d36f
feat: add p2p module deviation checking
gluax Jun 8, 2022
38debba
chore: clean up todo comment
gluax Jun 8, 2022
b6c7029
feat: sqlite logging
gluax Jun 14, 2022
073fbb9
feat: changes to p2p from admin audit
gluax Jun 16, 2022
df7dde2
feat: creates pairs if not yet made, as well as finish changes for th…
gluax Jun 17, 2022
582f3c1
fix: call right provider in evm call
gluax Jun 22, 2022
f475a9c
fix: p2p call deployOracle
mennatnaga Jun 27, 2022
5e4b197
refactor: check oracle status before deploying
mennatnaga Jun 27, 2022
c79cbdd
chore: add config files
mennatnaga Jun 28, 2022
0139f0a
chore: add createPeer.ts
mennatnaga Jun 28, 2022
4e5b540
fix: assert creator matches wallet public address before deployOracle
mennatnaga Jun 28, 2022
08b2134
fix: sort reports
mennatnaga Jun 29, 2022
8d26afe
fix: delete outdated (round) reports
mennatnaga Jun 29, 2022
5e7569f
fix: reselect leader when stuck
mennatnaga Jun 30, 2022
044fe07
fix: reuse old report data, but use relected round id
gluax Jun 30, 2022
1987fac
fix: require unique objects in Sets, sign fresh data
mennatnaga Jul 1, 2022
74a2f8d
fix: switch transmitters
mennatnaga Jul 1, 2022
3a46100
refactor: clear reports after each round, assume all sigs needed
mennatnaga Jul 2, 2022
339780c
feat: initial p2p fuzzing
gluax Jul 2, 2022
eaf8194
fix: merge upstream changes
gluax Jul 2, 2022
9ce0bab
refactor: allow transmitting less signatures
mennatnaga Jul 2, 2022
74285cb
chore: fuzzing code clean up
gluax Jul 2, 2022
fba3850
fix: non creator nodes now wait for pairs to be created instead of ex…
gluax Jul 2, 2022
0b2f65e
fix: fetch round before handling reports
mennatnaga Jul 4, 2022
3b74729
fix: rm fetching rounds, increase delay between queue batches
mennatnaga Jul 5, 2022
64f6285
node and report versioning
gluax Jul 6, 2022
7374005
Merge branch 'feature/p2p' of github.com:fluxprotocol/fpo-node into f…
gluax Jul 6, 2022
4a5fd0c
fix: resubmit unsuccessful rounds
mennatnaga Jul 6, 2022
220e769
feat(fuzzing): change workers out with clustering
gluax Jul 6, 2022
13f8b3a
fix: waiting on creator to deploy oracle & fix p2p versioning checks
gluax Jul 7, 2022
ab51a40
feat(fuzz): start working on random nodes disconnecting
gluax Jul 7, 2022
1cea595
fix: filter old unsuccessful sigs
mennatnaga Jul 7, 2022
86d001f
fix: fix p2p versioning issues, fix random disconnects in fuzzing
gluax Jul 10, 2022
8b1132d
Merge branch 'feature/p2p' of github.com:fluxprotocol/fpo-node into f…
gluax Jul 10, 2022
a03c0d5
fix: version checking on p2p, and p2p fuzzing version handling
gluax Jul 12, 2022
5aad90c
feat: fuzz report versions, add some new options for timings
gluax Jul 13, 2022
b7700c6
feat: add more fuzzing pair generation
gluax Jul 13, 2022
4f5860d
feat: clean up pair generation duplicates, config for interval and de…
gluax Jul 13, 2022
0c6f391
fix remove duplicate pairs
gluax Jul 13, 2022
18fe96c
fix: set roundId only after successful transmission
mennatnaga Jul 15, 2022
c43e7d7
fix: move p2p version check, temporarily disable version sharing in p…
gluax Jul 15, 2022
2b90f33
fix: wait for enough signatures
mennatnaga Jul 18, 2022
d839862
fix: check resolved rounds to avoid double transmit
mennatnaga Jul 20, 2022
60a2d4a
prevent transmitting 0s
gluax Jul 21, 2022
5ee7992
fix: transmit 0s and merge upstream
gluax Jul 21, 2022
c7097d5
feat: better log tracking for fuzzing runs
gluax Jul 22, 2022
04c5d37
fix: Use the correct local address for reconnecting
gluax Jul 22, 2022
15e99f9
fix: update peers to avoid choosing a disconnected leader, skip itera…
mennatnaga Jul 23, 2022
3c1c5b4
feat: re-enable version sending between p2p nodes
gluax Jul 26, 2022
7724101
chore: add postinstall command for moving fuzzer to separate repo
gluax Aug 1, 2022
a72fb25
fix: fix type module generation
gluax Aug 1, 2022
5f63d67
fix: include package json for types generation
gluax Aug 1, 2022
5f08e94
fix: need both js and .d.ts files for fuzzer
gluax Aug 1, 2022
cb7d059
fix: generate .d.ts files
gluax Aug 1, 2022
478ab34
fix: types generation including json files
gluax Aug 1, 2022
455133d
fix: exclude output dir from types generation
gluax Aug 1, 2022
9555f96
fix: add include back
gluax Aug 1, 2022
9059bfc
fix: json files included locally but not for deps
gluax Aug 1, 2022
284c466
fix: explicitly include all json files
gluax Aug 1, 2022
a88a7b6
refactor: adjust gasLimit for ganache
mennatnaga Aug 3, 2022
3cad051
feat: support multiple networks in p2p module
jamesondh Aug 3, 2022
d1a6cfe
refactor: skip less iterations
mennatnaga Aug 4, 2022
8130f17
feat: add getMinSignersForPair()
mennatnaga Aug 5, 2022
f21b6ca
refactor: remove fuzzer to another repo
gluax Aug 8, 2022
38c9905
feat: change nodes to just log they are out of date and not quit, as …
gluax Aug 8, 2022
2613804
fix: fix logging of versions in recieving
gluax Aug 8, 2022
54e0900
fix: trying to prevent version recieved from constantly going off.
gluax Aug 8, 2022
b9aeaf5
feat: clean up sending report version
gluax Aug 8, 2022
b8a1a22
feat: adding a trace to help debug why p2p-version is called so many …
gluax Aug 8, 2022
7a23de5
feat: more cleanup on version sending, still trying to debug multiple…
gluax Aug 8, 2022
2dfc5a3
refactor: waiting for sigs
mennatnaga Aug 9, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"cSpell.words": [
"unhandle"
]
}
13 changes: 11 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,26 +14,35 @@
"build": "tsc -p ."
},
"dependencies": {
"@chainsafe/libp2p-noise": "^5.0.3",
"@fluxprotocol/oracle-vm": "^2.0.0",
"big.js": "^6.1.1",
"bn.js": "^5.2.0",
"dotenv": "^15.0.0",
"ethers": "^5.5.4",
"libp2p": "^0.36.2",
"libp2p-mplex": "^0.10.7",
"libp2p-tcp": "^0.17.2",
"express": "^4.17.3",
"jsonpath-plus": "^6.0.1",
"lodash.topath": "^4.5.2",
"multiaddr": "^10.0.1",
"near-api-js": "^0.44.2",
"uint8arrays": "^3.0.0",
"web3": "^1.7.0",
"winston-daily-rotate-file": "^4.6.0",
"winston-transport-sentry-node": "^2.3.0"
},
"devDependencies": {
"@types/big.js": "^6.1.2",
"@types/bl": "^5.0.2",
"@types/debug": "^4.1.7",
"@types/express": "^4.17.13",
"@types/jest": "^27.4.0",
"@types/lodash.topath": "^4.5.6",
"@types/node": "^17.0.14",
"@types/node": "^17.0.21",
"jest": "^27.4.7",
"ts-jest": "^27.1.3",
"typescript": "^4.5.5"
"typescript": "^4.6.2"
}
}
1 change: 0 additions & 1 deletion src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import { PROJECT_NAME, PROJECT_VERSION } from './config';

async function main() {
logger.info(`🧙 Starting ${PROJECT_NAME} v${PROJECT_VERSION}`);

try {
const appConfig = await parseAppConfig();

Expand Down
6 changes: 6 additions & 0 deletions src/models/AppConfig.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import { CreateOptions } from "libp2p";

import { Healthcheck } from '../healthCheck/Healthcheck';
import { IModule } from './IModule';
import { ModuleConfig } from './IModule';
Expand All @@ -9,9 +11,13 @@ export interface AppConfig {
networks: INetwork[];
modules: IModule[];
jobs: IJob[];
p2p_node: CreateOptions;
peers_file: string;
}

export interface UnparsedAppConfig {
networks?: Partial<NetworkConfig>[];
modules?: Partial<ModuleConfig>[];
p2p_node: CreateOptions,
peers_file: string,
}
Loading