diff --git a/book/src/api-vc-endpoints.md b/book/src/api-vc-endpoints.md index 80eba7a0590..cb310c9f2c5 100644 --- a/book/src/api-vc-endpoints.md +++ b/book/src/api-vc-endpoints.md @@ -18,6 +18,7 @@ | [`POST /lighthouse/validators/mnemonic`](#post-lighthousevalidatorsmnemonic) | Create a new validator from an existing mnemonic. | | [`POST /lighthouse/validators/web3signer`](#post-lighthousevalidatorsweb3signer) | Add web3signer validators. | | [`GET /lighthouse/logs`](#get-lighthouselogs) | Get logs | +| [`GET /lighthouse/beacon/health`](#get-lighthousebeaconhealth) | Get health information for each connected beacon node. | The query to Lighthouse API endpoints requires authorization, see [Authorization Header](./api-vc-auth-header.md). @@ -816,3 +817,55 @@ logs emitted are INFO level or higher. } } ``` + +## `GET /lighthouse/beacon/health` + +Provides information about the sync status and execution layer health of each connected beacon node. +For more inormation about how to interpret the beacon node health, see [Fallback Health](./redundancy.md#fallback-health). + +### HTTP Specification + +| Property | Specification | +|-------------------|--------------------------------------------| +| Path | `/lighthouse/beacon/health` | +| Method | GET | +| Required Headers | [`Authorization`](./api-vc-auth-header.md) | +| Typical Responses | 200, 400 | + +Command: +```bash +DATADIR=/var/lib/lighthouse +curl -X GET http://localhost:5062/lighthouse/beacon/health \ + -H "Authorization: Bearer $(cat ${DATADIR}/validators/api-token.txt)" | jq + ``` + +### Example Response Body + +```json +{ + "data": { + "beacon_nodes": [ + { + "index": 0, + "endpoint": "http://localhost:5052", + "health": { + "user_index": 0, + "head": 10500000, + "optimistic_status": "No", + "execution_status": "Healthy", + "health_tier": { + "tier": 1, + "sync_distance": 0, + "distance_tier": "Synced" + } + } + }, + { + "index": 1, + "endpoint": "http://fallbacks-r.us", + "health": "Offline" + } + ] + } +} +``` diff --git a/book/src/redundancy.md b/book/src/redundancy.md index daf0eb4a5b4..3b488137231 100644 --- a/book/src/redundancy.md +++ b/book/src/redundancy.md @@ -39,9 +39,6 @@ There are a few interesting properties about the list of `--beacon-nodes`: earlier in the list. - *Synced is preferred*: the validator client prefers a synced beacon node over one that is still syncing. -- *Failure is sticky*: if a beacon node fails, it will be flagged as offline - and won't be retried again for the rest of the slot (12 seconds). This helps prevent the impact - of time-outs and other lengthy errors. > Note: When supplying multiple beacon nodes the `http://localhost:5052` address must be explicitly > provided (if it is desired). It will only be used as default if no `--beacon-nodes` flag is @@ -76,6 +73,22 @@ Prior to v3.2.0 fallback beacon nodes also required the `--subscribe-all-subnets now broadcast subscriptions to all connected beacon nodes by default. This broadcast behaviour can be disabled using the `--broadcast none` flag for `lighthouse vc`. +### Fallback Health + +Since v6.0.0, the validator client will be more aggressive in switching to a fallback node. To do this, +it uses the concept of "Health". Every slot, the validator client checks each connected beacon node +to determine which node is the "Healthiest". In general, the validator client will prefer nodes +which are synced, have synced execution layers and which are not currently optimisitically +syncing. + +Sync distance is separated out into 4 tiers: "Synced", "Small", "Medium", "Large". Nodes are then +sorted into tiers based onto sync distance and execution layer status. You can use the +`--beacon-nodes-sync-tolerances` to change how many slots wide each tier is. In the case where +multiple nodes fall into the same tier, user order is used to tie-break. + +To see health information for each connected node, you can use the +[`/lighthouse/beacon/health` API endpoint](./api-vc-endpoints.md#get-lighthousebeaconhealth). + ### Broadcast modes Since v4.6.0, the Lighthouse VC can be configured to broadcast messages to all configured beacon diff --git a/validator_client/http_api/src/lib.rs b/validator_client/http_api/src/lib.rs index b58c7ccec02..72aa5406bfc 100644 --- a/validator_client/http_api/src/lib.rs +++ b/validator_client/http_api/src/lib.rs @@ -414,10 +414,10 @@ pub fn serve( }, ); - // GET lighthouse/ui/fallback_health - let get_lighthouse_ui_fallback_health = warp::path("lighthouse") - .and(warp::path("ui")) - .and(warp::path("fallback_health")) + // GET lighthouse/beacon/health + let get_lighthouse_beacon_health = warp::path("lighthouse") + .and(warp::path("beacon")) + .and(warp::path("health")) .and(warp::path::end()) .and(block_service_filter.clone()) .then(|block_filter: BlockService| async move { @@ -1303,7 +1303,7 @@ pub fn serve( .or(get_lighthouse_validators_pubkey) .or(get_lighthouse_ui_health) .or(get_lighthouse_ui_graffiti) - .or(get_lighthouse_ui_fallback_health) + .or(get_lighthouse_beacon_health) .or(get_fee_recipient) .or(get_gas_limit) .or(get_graffiti)