Skip to content
/ SideLB Public

SideLB – Sidecar Load Balancer, a lightweight service load balancer that runs beside your actual Application. Espcially useful when dealing with decentralized services

License

Notifications You must be signed in to change notification settings

rsmvdl/SideLB

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SideLB – Sidecar Load Balancer, a lightweight service load balancer that runs beside your actual Application

SideLB manages traffic between your service (a server backend, …) and other services (databases, caching clusters, …). It provides robust and efficient load balancing for backends that require seamless integration with decentralized services, no matter the protocol.

SideLB supports infrastructures built for decentralized applications. E.g., you can run it in the same container as your application, this is actually the idea.

SideLB is written in Rust and primarily based on the tokio library.

Key features

  • Perform DNS and reverse DNS (rDNS) resolution to discover both IPv4 and IPv6 members of a domain. For example, in a decentralized CockroachDB or YugabyteDB setup, a domain like db.example.com may resolve to multiple IP addresses (e.g., 10 IPv4 and 10 IPv6 addresses). SideLB leverages DNS/rDNS resolution to intelligently distribute traffic across all available servers.
  • Group multiple IP addresses belonging to the same server and treats them as a single entity. This prevents server with multiple public IPs from being overburdened. (Note that DNS/rDNS must be setup correctly!)
  • No protocol overhead. SideLB is purpose-built for TCP/UDP traffic and does not handle HTTP or other service-specific protocols directly. This makes it particularly suited for routing database queries and other low-level service communications.
  • Continuously monitor the availability of each server, ensuring traffic is only routed to healthy servers. It supports two key load balancing algorithms:
    1. round-robin: Evenly distributes traffic across all available servers, and
    2. least-connections: Routes traffic to the server with the fewest active connections.

Setup structure example

                                                    +----------------------+
                                                    |       Request        |
                                                    +----------------------+
                                                               |
                                                               v
                                                    +----------------------+
                                                    | Backend Application  |
                                                    +----------------------+
                                                               |
                                                               v
                                                    +----------------------+
                                                    |         SideLB       |   
                                                    |    127.0.0.1:5432    |
+----------------------------------------------------------------------------------------------------------------------------+ -> db.example.com
             |                        |                        |                        |                        |
             v                        v                        v                        v                        v
+------------------------+------------------------+------------------------+------------------------+------------------------+
|     CockroachDB 1      |     CockroachDB 2      |     CockroachDB 3      |     CockroachDB 4      |     CockroachDB 5      |
|   100.100.100.101:5432 |   100.100.100.102:5432 |   100.100.100.103:5432 |   100.100.100.104:5432 |   100.100.100.105:5432 |
|     2001:db8::1:5432   |     2001:db8::2:5432   |     2001:db8::3:5432   |     2001:db8::4:5432   |     2001:db8::5:5432   |
+------------------------+------------------------+------------------------+------------------------+------------------------+



Usage

Your backend connects to SideLB, which then forwards the traffic to the appropriate backend services. You can specify either static IP addresses for backend services or a ring domain (ring_domain) for dynamic DNS resolution.

Using a ring domain:

sidelb 127.0.0.1:5432 ring_domain=db.example.com:5432 mode=round-robin

In this example:

  • 127.0.0.1:5432 is the address where SideLB listens for incoming connections from your backend.
  • ring_domain=db.example.com allows SideLB to dynamically resolve backend IPs using DNS, e.g. your CockroachDB cluster.
  • mode=round-robin ensures that traffic is evenly distributed across all resolved backend service members.
  • proto=tcp/udp Optional, sets the desired protocol to use, you can select between TCP and UDP, TCP is the default if not set.

Using static IP addresses:

sidelb 127.0.0.1:5432 100.100.100.103:5432 100.100.100.104:5432 mode=least-connections

Here, SideLB forwards traffic from 127.0.0.1:5432 to 100.100.100.103:5432 and 100.100.100.104:5432.

Additionally, you can also manually select the protocol you want to load balance (TCP/UDP), just simply do:

sidelb 127.0.0.1:5432 100.100.100.103:5432 100.100.100.104:5432 mode=least-connections proto=tcp

or using the ring_domain respectively

sidelb 127.0.0.1:5432 ring_domain=db.example.com:5432 mode=round-robin proto=tcp

and so on

Known Limitations

  • **Load balancing is only relative with SideLB, as most likely many containers or servers consuming a service like a Database and SideLB instances don't communicate with each other at all ...

  • SSL/TLS Name Validation: When using SSL/TLS between the backend and the target servers, name validation may need to be disabled or explicitly add 127.0.0.1 to your certs. Since the backend communicates with SideLB over 127.0.0.1, which then forwards the request to the target server, the original domain name might not match the SSL certificate. This can lead to SSL name validation errors unless explicitly turned off.

    For example, when you're using the Django framework you would set the following for your Database settings:

      'sslmode': 'verify-ca',  # Changed from 'verify-full' to 'verify-ca' to skip hostname verification
    

How to build?

Compiling SideLB is fairly easy, you simply run the included trigger.sh script locally, this will spin-up an Ubuntu 24.04 container compiling the software for you (Docker required), the final executable binary is than available at the /build folder as the compiled binary is simply getting copied from the Docker Container to your local filesystem.

./trigger.sh

Implementation example inside a container

Many developers that build containers using supervisord to manage processes spawned inside there container(s). The same way you can do it with SideLB, simply include the binary at e.g. /usr/bin of you container, make it executable and start an instance like so:

[program:SideLB]
user=my_user
autostart=true
autorestart=true
command=sidelb_flow
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stdout_logfile_backups=0
redirect_stderr=true
killasgroup=true
stopasgroup=true
priority=100

[program:ExampleApp]
user=my_user
autostart=true
autorestart=true
command=app_exec_command
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stdout_logfile_backups=0
redirect_stderr=true
killasgroup=true
stopasgroup=true
priority=200

the sidelb_flow script is just an example here, you can also implement this otherwise, but this is how I did it in combination with env variables:

#!/usr/bin/env bash

function sidelb_exec {
echo "Starting SideLB"

# Use environment variables to form the sidelb command
sidelb 127.0.0.1:${DATABASE_PORT} ring_domain="${DATABASE_ENDPOINT}:${DATABASE_PORT}" mode=${SIDELB_MODE}
}

sidelb_exec

With this setup, your application connects to the database through 127.0.0.1:${DATABASE_PORT}, e.g. port 5432 for postgres. SideLB then resolves the ring_domain, which points to the actual database service consisting of multiple IPs, and forwards the request to one of these resolved IP addresses using the selected load balancing algorithm.

Who Should Use SideLB?

  • **Developers working with microservices or decentralized services like Redis, CockroachDB or YugabyteDB, where traffic needs to be efficiently routed across multiple service nodes.
  • **Teams deploying applications where an embedded load balancer is required for efficient traffic management like communicating with decentralized services.
  • **Environments where external load balancers are unnecessary or add unwanted complexity, as SideLB provides a lightweight, embedded alternative.

Requirements

  • **A Domain name under your control
  • **A service you want to reach using SideLB, e.g. CockRoachDB or similar that exists on the Public Web or private infrastructure

Final tough's and notices

I developed SideLB as part of a larger project that heavily relays on decentralization, especially for the Database communication. This way I wasn't in need anymore to manage and/or rent load balancer resources on platforms like AWS, Digitalocean, Google etc. as the application itself is now able to load balance connections by itself with a certain service endpoint.

If you set up a decentralized service like a CockRoachDB or similar, consider to split them up into Geo-Zones on DNS side, like:

  • us.database.com (100.100.100.101, 100.100.100.102, 100.100.100.103)
    • -> node01.us.my-service.com - 100.100.100.101
    • -> node02.us.my-service.com - 100.100.100.102
    • -> node03.us.my-service.com - 100.100.100.103
  • de.database.com (101.100.100.101, 101.100.100.102, 101.100.100.103)
    • -> node01.de.my-service.com - 101.100.100.101
    • -> node02.de.my-service.com - 101.100.100.102
    • -> node03.de.my-service.com - 101.100.100.103
  • fr.database.com (102.100.100.101, 102.100.100.102, 102.100.100.103)
    • -> node01.fr.my-service.com - 102.100.100.101
    • -> node02.fr.my-service.com - 102.100.100.102
    • -> node03.fr.my-service.com - 102.100.100.103

Ensure that each node’s IP points to its corresponding geographic subdomain like us.database.com ... This way you can make sure that your application always communicates with a group of servers that are geographically near to each other.

About

SideLB – Sidecar Load Balancer, a lightweight service load balancer that runs beside your actual Application. Espcially useful when dealing with decentralized services

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published