Self hosting backend service instruction

docs/user-guides/bitcoin-staking-testnet-4/backend-deployment/deployment-overview.md

@@ -0,0 +1,46 @@
+---
+id: deployment-overview
+sidebar_label: Deployment Overview
+hide_table_of_contents: true
+
+---
+# Overview of Bitcoin Staking backend deployment
+
+![Overview of Bitcoin Staking backend deployment](images/phase-1-overview.png)
+
+The Babylon Bitcoin Staking system comprises of the following components:
+
+- [**BTC Staking Indexer**](../backend-deployment/services/staking-indexer.md):
+ Parses BTC blocks for valid staking, unbonding,
+and withdrawal transactions, and forwards relevant events to a queueing system,
+while also persisting them to an on-disk key-value storage.
+- [**RabbitMQ**](./infra/rabbitmq.md):
+Houses a set of queues containing BTC Staking transactions.
+- [**Staking API Service**](./services/staking-api.md):
+Consumes BTC Staking transactions from the RabbitMQ queues
+and stores them in a central data store,
+additionally accepting unbonding requests.
+- [**MongoDB**](./infra/mongodb.md): Stores BTC Staking transaction data.
+- [**Staking Expiry Checker**](./services/staking-expiry-checker.md):
+Periodically checks MongoDB for expired BTC Stake Delegations and Unbondings.
+- [**Unbonding Pipeline**](./services/staking-expiry-checker.md):
+Forwards unbonding requests for signing to a Covenant Emulator committee
+and submits them to BTC network
+- [**Staking Dashboard**](https://github.com/babylonchain/btc-staking-dashboard):
+UI that allows for creating BTC Staking transactions,
+connects to the API to retrieve information about the system and historical delegations.
+- [**Covenant Signer**](https://github.com/babylonchain/covenant-signer/blob/dev/docs/deployment.md#covenant-signer-setup-deployment):
+Receives unbonding transactions and returns the same
+transactions signed by the covenant emulator's key.
+- [**Bitcoin Full Node**](./infra/bitcoind.md):
+Operates an emulated regtest Bitcoin network.
+- [**Bitcoin Offline Wallet**](https://github.com/babylonchain/covenant-signer/blob/dev/docs/deployment.md#3-bitcoind-offline-wallet-operations):
+Stores the Covenant Signer member keys
+and signs unbonding transactions forwarded by the Covenant Signer.
+- [**A Global Configuration file**](./global-system-configuration.md)
+that contains system-wide parameters pertinent
+to the processed Staking transactions.
+- [**A Finality Provider config file**](./global-system-configuration.md)
+that associates finality provider
+BTC public keys with additional information about them such
+as their moniker, commission etc.

docs/user-guides/bitcoin-staking-testnet-4/backend-deployment/global-system-configuration.md

@@ -0,0 +1,104 @@
+---
+id: global-params
+sidebar_label: Global System Configuration
+hide_table_of_contents: false
+---
+# Global System Configuration
+
+## 1. Staking Parameters
+
+The Global Configuration includes versioned governance parameters
+that greatly affect the behaviour of the system.
+
+Detailed information can be found
+[here](https://github.com/babylonchain/networks/tree/391cecff2569a57a26317e93484c5690b6ce6540/bbn-test-4/parameters).
+
+Depending on the network, you can download the corresponding version:
+
+- [Testnet](https://github.com/babylonchain/networks/blob/391cecff2569a57a26317e93484c5690b6ce6540/bbn-test-4/parameters/global-params.json)
+
+## 2. Finality Providers
+
+Finality providers can accept delegations from BTC stakers.
+For more details, please visit
+[this link](https://github.com/babylonchain/networks/tree/524247c418af07beb99a291a1686413c23d22553/bbn-test-4/finality-providers).
+
+Finality provider information registry stores additional information
+such as the finality provider's moniker, website, and identity.
+
+There are many registries that might exist, and it is up to the staking provider
+to utilize a source of information that best fits their requirements.
+This document uses the registry information maintained by Babylon.
+
+### 2.1 Accessing Individual Finality Provider Information
+
+Finality provider information is available in individual JSON files within the
+finality provider registry. You can find these files in the following directory,
+depending on the network:
+
+- [Testnet](https://github.com/babylonchain/networks/tree/524247c418af07beb99a291a1686413c23d22553/bbn-test-4/finality-providers/registry)
+
+Each JSON file contains information in the following format:
+
+```JSON
+{
+  "description": {
+    "moniker": "<moniker>",
+    "identity": "<identity>",
+    "website": "<website>",
+    "security_contact": "<security_contact>",
+    "details": "<details>"
+  },
+  "btc_pk": "<eots_btc_pk>",
+  "commission": "<commission_decimal>",
+  "deposit": {
+    "tx_hash": "tx_hash",
+    "signed_tx": "signed_tx_hex"
+  }
+}
+```
+
+### 2.2 Generating Concatenated Finality Provider Information
+
+The concatenated Finality Provider information file is essential for backend
+services that require a unified view of all finality provider information.
+
+To generate a concatenated JSON file from the individual JSON files,
+use the following reference script.
+Depending on the network, you can change the corresponding directory `DIR`:
+
+- testnet: `export DIR="./networks/bbn-test-4/finality-providers/registry"`
+
+```bash
+echo '
+# Clone the repository
+git clone [email protected]:babylonchain/networks.git
+
+# Output files
+OUTPUT="temp.json"
+FINAL="finality-providers.json"
+
+# Start the JSON object and array
+echo "{" > $OUTPUT
+echo "\"finality_providers\": [" >> $OUTPUT
+
+# Iterate over JSON files and append them to the output file
+for file in "$DIR"/*.json; do
+    cat "$file" >> $OUTPUT
+    echo "," >> $OUTPUT
+done
+
+# Remove the last comma and close the JSON array and object
+truncate -s-2 $OUTPUT
+echo "]" >> $OUTPUT
+echo "}" >> $OUTPUT
+
+# Beautify the final JSON output
+cat $OUTPUT | jq . > $FINAL
+' > generate_finality_providers.sh && bash generate_finality_providers.sh
+```
+
+This script clones the Babylon networks repository,
+navigates to the specified directory,
+iterate over JSON files and
+append them to the output file containing all Finality provider information.

docs/user-guides/bitcoin-staking-testnet-4/backend-deployment/hardware-requirements.md

@@ -0,0 +1,66 @@
+---
+id: hardware-requirements
+sidebar_label: Hardware Requirements
+hide_table_of_contents: true
+
+---
+# Hardware requirements
+
+## Required Infrastructure Services
+
+### MongoDB
+
+:::info Note
+
+For the most accurate and up-to-date hardware recommendations,
+please refer to the official
+[documentation](https://www.mongodb.com/docs/cloud-manager/tutorial/provisioning-prep/#deployment-prerequisites)
+
+:::
+
+A replica set cluster with at least 3 nodes is required
+to ensure data redundancy and high availability.
+
+Hardware Requirements:
+
+- CPU: Multi-core processors.
+- Memory: Minimum 4GB RAM per node.
+- Storage: SSDs recommended for better performance.
+
+The MongoDB Agent must be installed only on 64-bit architectures.
+
+### RabbitMQ
+
+:::info Note
+
+For the most accurate and up-to-date hardware recommendations,
+please refer to the official [documentation](https://www.rabbitmq.com/docs/production-checklist)
+
+:::
+
+For RabbitMQ, a multi-node cluster with at least 3 nodes is required
+to ensure message durability and high availability.
+
+Hardware Requirements:
+
+- CPU: Multi-core processors.
+- Memory: Minimum 4GB RAM per node.
+- Storage: SSDs recommended for better performance.
+Network: High-speed, low-latency network connection.
+
+### Bitcoind
+
+Hardware Requirements:
+
+- CPU: Multi-core processor.
+- Memory: Minimum 8GB RAM.
+- Storage: SSDs with at least 800GB of space to accommodate the full Bitcoin blockchain.
+
+## BTC Staking Backend Services
+
+Each of the backend services should be hosted on separate servers
+to ensure optimal performance and isolation.
+Below are the suggested hardware requirements:
+
+- CPU: Multi-core processor (4 cores minimum)
+- Memory: Minimum 4GB RAM, recommended 8GB RAM

docs/user-guides/bitcoin-staking-testnet-4/backend-deployment/infra/bitcoind.md

@@ -0,0 +1,140 @@
+---
+id: bitcoind
+sidebar_label: Bitcoind
+hide_table_of_contents: false
+---
+# Bitcoin Node Setup
+
+## 1. Bitcoin Core Setup
+
+Download and install the bitcoin binaries according to your operating system
+from the official
+[Bitcoind Core registry](https://bitcoincore.org/bin/bitcoin-core-26.0/).
+All programs in this guide are compatible with version 26.0.
+
+## 2. Configuration
+
+bitcoind is configured through a main configuration file named bitcoin.conf.
+
+Depending on the operating system,
+the configuration file should be placed under the corresponding path:
+
+- MacOS: `/Users/<username>/Library/Application Support/Bitcoin`
+- Linux: `/home/<username>/.bitcoin`
+- Windows: `C:\Users\<username>\AppData\Roaming\Bitcoin`
+
+Both servers can utilize the following base parameter skeleton
+(adapted for the BTC signet network):
+
+```text
+# Accept command line and JSON-RPC commands
+server=1
+# Enable transaction indexing
+txindex=1
+# RPC server settings
+rpcuser=<rpc-username>
+rpcpassword=<rpc-password>
+# Optional: In case of non-mainnet BTC node,
+# the following two lines specify the network that your
+# node will operate; for this example, utilizing signet
+signet=1
+[signet]
+# Port your bitcoin node will listen for incoming requests;
+# below port is the canonical port for signet,
+# for mainnet, typically 8332 is used
+rpcport=38332
+# Address your bitcoin node will listen for incoming requests
+rpcbind=0.0.0.0
+# Optional: Needed for remote node connectivity
+rpcallowip=0.0.0.0/0
+```
+
+Notes:
+
+- Instead of hardcoding the RPC server password (`rpcpassword`) in the config,
+it is recommended to generate its salted hash and use the rpcauth field instead.
+To generate the salted hash, you can use
+this [platform](https://jlopp.github.io/bitcoin-core-rpc-auth-generator/)
+as reference - however, the salting operation should be executed locally.
+The resulting config value will look like this:
+
+```text
+rpcauth=<rpc-password-salted-hash>
+```
+
+- In case you will be connecting to BTC Mainnet network,
+ensure to remove the following config skeleton lines:
+
+```text
+signet=1
+[signet]
+```
+
+## 3. Start bitcoind
+
+### 3.1 Start the Bitcoin daemon
+
+```bash
+bitcoind
+```
+
+### 3.2 Verify bitcoind is Running
+
+Check the status of bitcoind.
+
+```bash
+bitcoin-cli getblockchaininfo
+```
+
+You should see information about the blockchain,
+indicating that `bitcoind` is running correctly.
+
+## 4. Create a systemd Service for bitcoind (Optional)
+
+### 4.1 Create systemd service definition
+
+Run the following command, replacing `your_username` with your actual username:
+
+```bash
+cat <<EOF | sudo tee /etc/systemd/system/bitcoind.service
+[Unit]
+Description=Bitcoin daemon
+After=network.target
+
+[Service]
+ExecStart=/usr/local/bin/bitcoind -conf=/home/your_username/.bitcoin/bitcoin.conf
+ExecStop=/usr/local/bin/bitcoin-cli stop
+User=your_username
+Restart=always
+
+[Install]
+WantedBy=multi-user.target
+EOF
+```
+
+### 4.2 Reload systemd manager configuration
+
+```bash
+sudo systemctl daemon-reload
+```
+
+### 4.3 Enable the service to start on boot
+
+```bash
+sudo systemctl enable bitcoind.service
+```
+
+### 4.4 Start the service
+
+```bash
+sudo systemctl start bitcoind.service
+```
+
+## 5. Monitoring
+
+The bitcoind server availability can be polled through
+[Prometheus Blackbox Exporter](https://github.com/prometheus/blackbox_exporter).
+
+Bitcoin-specific Prometheus metrics can also be exposed
+by utilizing any open-source Prometheus bitcoind exporter
+([example](https://github.com/jvstein/bitcoin-prometheus-exporter?tab=readme-ov-file)).