Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multi-database support to EchoVault #83

Merged
merged 45 commits into from
Jul 2, 2024
Merged

Conversation

kelvinmwinuka
Copy link
Collaborator

This change adds key name spacing using logical database to EchoVault. These name spaces are called databases and can be selected using indexes.

Features

  • Key name spaces can be created in both standalone mode and RAFT cluster mode.
  • Key name spaces are supported in AOF files and snapshot files. Multiple databases will be saved within the same AOF and snapshot file.
  • Name spaces are restored from disk upon startup.
  • New commands to interact with databases.

Client-Server Commands

HELLO

HELLO [protover [AUTH username password] [SETNAME clientname]]

The hello commands returns information about the client. The AUTH options allow the client to authenticate itself. The SETNAME sets the alias for the current connection.

Example response with protocol 2:

1) server
2) "echovault"
3) version
4) "0.8.0"
5) proto
6) (integer) 2
7) id
8) (integer) 1
9) mode
10) "standalone"
11) role
12) "master"
13) modules
14)  1) "acl"
       2) "admin"
       3) "connection"
       4) "generic"
       5) "hash"
       6) "list"
       7) "pubsub"
       8) "set"
       9) "sortedset"
      10) "string"

Example response with protocol 3:

1# server => "echovault"
2# version => "0.8.0"
3# proto => (integer) 3
4# id => (integer) 1
5# mode => "standalone"
6# role => "master"
7# modules => 
    1) "acl"
    2) "admin"
    3) "connection"
    4) "generic"
    5) "hash"
    6) "list"
    7) "pubsub"
    8) "set"
    9) "sortedset"
   10) "string"

SELECT

SELECT index

Change the logical database that the current connection is operating from.

SWAPDB

SWAPDB index1 index2

This command swaps two databases,
so that immediately all the clients connected to a given database will see the data of the other database,
and the other way around.

FLUSHALL

FLUSHALL

Delete all the keys in all the existing databases. This command is always synchronous.

FLUSHDB

FLUSHDB

Delete all the keys in the currently selected database. This command is always synchronous.

Embedded API

SetProtocol

echovault.SetProtocol(protocol int) error

This methods sets the protocol that is expected by responses to embedded API calls. This is relevant for the ExecuteCommand methods which returns raw RESP to the caller.

SelectDB

echovault.SelectDB(database int) error

This method sets the database that the embedded client is operating from.

SwapDBs

echovault.SwapDBs(database1, database2 int)

SwapDBs swaps every TCP client connection from database1 over to database2. It also swaps every TCP client connection from database2 over to database1. This only affects TCP connections, it does not swap the logical database currently being used by the embedded API.

Flush

echovault.Flush(database int)

Flush flushes all the data from the database at the specified index. When -1 is passed, all the logical databases are cleared.

kelvinmwinuka and others added 30 commits July 1, 2024 06:04
…and standalone persistence initialisation until their implementations support databases as well.
… done. Updated function signatures in handler params for built-in commands and for modules. Disabled snapshots and snapshot restoration until a multi-database implementation is done.
…termine which connection info to use into the handleCommand function.
Load modules build from Dockerfile.dev into instances from docker-compose.yml.
Fixed user loading from config file in ACL module.
…d echo command to the list of commands that are skipped on ACL authorization.
…unction. Implemented SELECT command to allow TCP connections to select a different database.
…horization in acl module. Added tests for HELLO and SELECT client-server commands
Added multi-database support for preamble snapshots and restoration in preamble engine.
Added multi-database support for aof engine.
Removed buffered channel from log and aof engines. Command logging is now synchronous.
Added multi-database support for the raft layer's data replication.
Added test cases for incorrect commands length for INCR command.
Fixed error checking on INCR commands test. RESP errors will be contained in the response object of the ReadValue method. The error object only contains an error when ReadValue failes.
Rebased with main branch.
@kelvinmwinuka kelvinmwinuka added the enhancement New feature or request label Jul 1, 2024
@kelvinmwinuka kelvinmwinuka self-assigned this Jul 1, 2024
@kelvinmwinuka kelvinmwinuka linked an issue Jul 1, 2024 that may be closed by this pull request
Copy link

codecov bot commented Jul 1, 2024

Codecov Report

Attention: Patch coverage is 70.73171% with 204 lines in your changes missing coverage. Please review.

Project coverage is 79.88%. Comparing base (d7bcc3a) to head (85e8a68).

Files Patch % Lines
echovault/keyspace.go 28.27% 104 Missing ⚠️
internal/aof/log/store.go 59.25% 16 Missing and 6 partials ⚠️
echovault/echovault.go 78.04% 17 Missing and 1 partial ⚠️
internal/modules/connection/commands.go 88.03% 7 Missing and 7 partials ⚠️
internal/aof/engine.go 35.00% 10 Missing and 3 partials ⚠️
internal/aof/preamble/store.go 62.96% 9 Missing and 1 partial ⚠️
internal/snapshot/snapshot.go 50.00% 5 Missing and 2 partials ⚠️
internal/modules/connection/utils.go 82.35% 4 Missing and 2 partials ⚠️
internal/modules/acl/acl.go 69.23% 3 Missing and 1 partial ⚠️
internal/modules/generic/commands.go 93.18% 1 Missing and 2 partials ⚠️
... and 2 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #83      +/-   ##
==========================================
- Coverage   80.39%   79.88%   -0.51%     
==========================================
  Files          48       50       +2     
  Lines        8304     8692     +388     
==========================================
+ Hits         6676     6944     +268     
- Misses       1144     1246     +102     
- Partials      484      502      +18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…ng snapshot on startup. Implemented tests for HELLO command in echovault package. Implemented tests for GetServerInfo in echovault package.
@kelvinmwinuka kelvinmwinuka merged commit b794a33 into main Jul 2, 2024
1 of 3 checks passed
@kelvinmwinuka kelvinmwinuka deleted the feature/multi-database branch September 12, 2024 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement HELLO command
2 participants