-
-
Notifications
You must be signed in to change notification settings - Fork 511
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(scylladb): add scylladb provider #2919
base: main
Are you sure you want to change the base?
Changes from all commits
672e6a7
41ed131
0003747
6a472cb
264d462
680c279
a995a1c
8f33187
7559c61
75de222
37ca8fc
7f7a476
e780e4f
95ca13f
01a4fef
4388885
c0f97ce
82e5352
cf070be
90caba2
aabf4d9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,131 @@ | ||||||
# ScyllaDB | ||||||
|
||||||
Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||||||
|
||||||
## Introduction | ||||||
|
||||||
The Testcontainers module for ScyllaDB, a NoSQL database fully compatible with Apache Cassandra and DynamoDB, allows you | ||||||
to create a ScyllaDB container for testing purposes. | ||||||
|
||||||
## Adding this module to your project dependencies | ||||||
|
||||||
Please run the following command to add the ScyllaDB module to your Go dependencies: | ||||||
|
||||||
```shell | ||||||
go get github.com/testcontainers/testcontainers-go/modules/scylladb | ||||||
``` | ||||||
|
||||||
## Usage example | ||||||
|
||||||
<!--codeinclude--> | ||||||
[Creating a ScyllaDB container](../../modules/scylladb/examples_test.go) inside_block:ExampleRun | ||||||
<!--/codeinclude--> | ||||||
|
||||||
## Module Reference | ||||||
|
||||||
### Run function | ||||||
|
||||||
- Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||||||
|
||||||
The ScyllaDB module exposes one entrypoint function to create the ScyllaDB container, and this function receives three parameters: | ||||||
|
||||||
```golang | ||||||
func Run(ctx context.Context, img string, opts ...testcontainers.ContainerCustomizer) (*ScyllaDBContainer, error) | ||||||
``` | ||||||
|
||||||
- `context.Context`, the Go context. | ||||||
- `string`, the Docker image to use. | ||||||
- `testcontainers.ContainerCustomizer`, a variadic argument for passing options. | ||||||
|
||||||
!!! info | ||||||
By default, we add the `--developer-mode=1` flag to the ScyllaDB container to disable the various checks Scylla | ||||||
performs. | ||||||
Also In scenarios in which static partitioning is not desired - like mostly-idle cluster without hard latency | ||||||
requirements, the --overprovisioned command-line option is recommended. This enables certain optimizations for ScyllaDB | ||||||
to run efficiently in an overprovisioned environment. You can change it by using the `WithCustomCommand` function. | ||||||
|
||||||
### Container Options | ||||||
|
||||||
When starting the ScyllaDB container, you can pass options in a variadic way to configure it. | ||||||
|
||||||
#### Image | ||||||
|
||||||
If you need to set a different ScyllaDB Docker image, you can set a valid Docker image as the second argument in the | ||||||
`Run` function. Eg: | ||||||
|
||||||
```golang | ||||||
scylladb.Run(context.Background(), "scylladb/scylla:6.2.1") | ||||||
// OR | ||||||
scylladb.Run(context.Background(), "scylladb/scylla:5.6") | ||||||
``` | ||||||
|
||||||
{% include "../features/common_functional_options.md" %} | ||||||
|
||||||
#### With Database Configuration File (scylla.yaml) | ||||||
|
||||||
- Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||||||
|
||||||
In the case you have a custom config file for ScyllaDB, it's possible to copy that file into the container before it's | ||||||
started, using the `WithConfigFile(cfgPath string)` function. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. suggestion: let's use the |
||||||
|
||||||
<!--codeinclude--> | ||||||
[With Config YAML](../../modules/scylladb/examples_test.go) inside_block:runScyllaDBContainerWithConfigFile | ||||||
<!--/codeinclude--> | ||||||
!!!warning | ||||||
You should provide a valid ScyllaDB configuration file when using the function, otherwise the container will fail to | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. suggestion: see above
Suggested change
|
||||||
start. The configuration file should be a valid YAML file and follows | ||||||
the [ScyllaDB configuration file](https://github.com/scylladb/scylladb/blob/master/conf/scylla.yaml). | ||||||
|
||||||
#### With Shard Awareness | ||||||
|
||||||
- Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||||||
|
||||||
If you want to test ScyllaDB with shard awareness, you can use the `WithShardAwareness` function. This function will | ||||||
configure the ScyllaDB container to use the `19042` port and ask the container to wait until the port is ready. | ||||||
|
||||||
<!--codeinclude--> | ||||||
[With Shard Awareness](../../modules/scylladb/examples_test.go) inside_block:runScyllaDBContainerWithShardAwareness | ||||||
<!--/codeinclude--> | ||||||
|
||||||
#### With Alternator (DynamoDB Compatible API) | ||||||
|
||||||
- Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||||||
|
||||||
If you want to test ScyllaDB with the Alternator API, you can use the `WithAlternator` function. This function will | ||||||
configure the ScyllaDB container to use the port any port you want and ask the container to wait until the port is | ||||||
ready. | ||||||
By default, you can choose the port `8000`. | ||||||
|
||||||
<!--codeinclude--> | ||||||
[With Alternator API](../../modules/scylladb/examples_test.go) inside_block:runScyllaDBContainerWithAlternator | ||||||
<!--/codeinclude--> | ||||||
|
||||||
#### With Custom Commands | ||||||
|
||||||
- Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||||||
|
||||||
If you need to pass any flag to the ScyllaDB container, you can use the `WithCustomCommand` function. This also rewrites | ||||||
predefined commands like `--developer-mode=1`. You can check | ||||||
the [ScyllaDB Docker Best Practices](https://opensource.docs.scylladb.com/stable/operating-scylla/procedures/tips/best-practices-scylla-on-docker.html) for more information. | ||||||
|
||||||
<!--codeinclude--> | ||||||
[With Custom Commands](../../modules/scylladb/examples_test.go) inside_block:runScyllaDBContainerWithCustomCommands | ||||||
<!--/codeinclude--> | ||||||
|
||||||
### Container Methods | ||||||
|
||||||
The ScyllaDB container exposes the following methods: | ||||||
|
||||||
#### ConnectionHost | ||||||
|
||||||
- Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||||||
|
||||||
This method returns the host and port of the ScyllaDB container, depending on the feature you want. | ||||||
If you just want to test it with a single node and a single core, you can use port `9042`. However, if you're planning | ||||||
to | ||||||
more than one core, you should use the **shard-awareness** port `19042`. If you're planning to use the **Alternator**? | ||||||
API, you should use the port you select in the `WithAlternator` function. | ||||||
|
||||||
<!--codeinclude--> | ||||||
[Get connection host](../../modules/scylladb/examples_test.go) inside_block:BaseConnectionHost | ||||||
<!--/codeinclude--> |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
include ../../commons-test.mk | ||
|
||
.PHONY: test | ||
test: | ||
$(MAKE) test-scylladb |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,226 @@ | ||
package scylladb_test | ||
|
||
import ( | ||
"context" | ||
"fmt" | ||
"log" | ||
"path/filepath" | ||
|
||
"github.com/gocql/gocql" | ||
|
||
"github.com/testcontainers/testcontainers-go" | ||
"github.com/testcontainers/testcontainers-go/modules/scylladb" | ||
) | ||
|
||
func ExampleRun_withCustomCommands() { | ||
ctx := context.Background() | ||
|
||
// runScyllaDBContainerWithCustomCommands { | ||
scyllaContainer, err := scylladb.Run(ctx, | ||
"scylladb/scylla:6.2", | ||
scylladb.WithCustomCommands(map[string]string{ | ||
"--memory": "1G", | ||
"--smp": "2", | ||
}), | ||
) | ||
// } | ||
defer func() { | ||
if err := testcontainers.TerminateContainer(scyllaContainer); err != nil { | ||
log.Printf("failed to terminate container: %s", err) | ||
} | ||
}() | ||
if err != nil { | ||
log.Printf("failed to start container: %s", err) | ||
return | ||
} | ||
|
||
state, err := scyllaContainer.State(ctx) | ||
if err != nil { | ||
log.Printf("failed to get container state: %s", err) | ||
return | ||
} | ||
|
||
fmt.Println(state.Running) | ||
|
||
connectionHost, err := scyllaContainer.ConnectionHost(ctx, 9042) // Non ShardAwareness port | ||
if err != nil { | ||
log.Printf("failed to get connection host: %s", err) | ||
return | ||
} | ||
|
||
runGoCQLExampleTest(connectionHost) | ||
|
||
// Output: | ||
// true | ||
} | ||
|
||
func ExampleRun_withAlternator() { | ||
ctx := context.Background() | ||
|
||
// runScyllaDBContainerWithAlternator { | ||
scyllaContainer, err := scylladb.Run(ctx, | ||
"scylladb/scylla:6.2", | ||
scylladb.WithAlternator(8080), // Choose which port to use on Alternator | ||
) | ||
// } | ||
defer func() { | ||
if err := testcontainers.TerminateContainer(scyllaContainer); err != nil { | ||
log.Printf("failed to terminate container: %s", err) | ||
} | ||
}() | ||
if err != nil { | ||
log.Printf("failed to start container: %s", err) | ||
return | ||
} | ||
|
||
state, err := scyllaContainer.State(ctx) | ||
if err != nil { | ||
log.Printf("failed to get container state: %s", err) | ||
return | ||
} | ||
|
||
fmt.Println(state.Running) | ||
|
||
_, err = scyllaContainer.ConnectionHost(ctx, 8080) // Alternator port | ||
if err != nil { | ||
log.Printf("failed to get connection host: %s", err) | ||
return | ||
} | ||
|
||
// Output: | ||
// true | ||
} | ||
|
||
func ExampleRun_withShardAwareness() { | ||
ctx := context.Background() | ||
|
||
// runScyllaDBContainerWithShardAwareness { | ||
scyllaContainer, err := scylladb.Run(ctx, | ||
"scylladb/scylla:6.2", | ||
scylladb.WithShardAwareness(), | ||
) | ||
// } | ||
defer func() { | ||
if err := testcontainers.TerminateContainer(scyllaContainer); err != nil { | ||
log.Printf("failed to terminate container: %s", err) | ||
} | ||
}() | ||
if err != nil { | ||
log.Printf("failed to start container: %s", err) | ||
return | ||
} | ||
|
||
state, err := scyllaContainer.State(ctx) | ||
if err != nil { | ||
log.Printf("failed to get container state: %s", err) | ||
return | ||
} | ||
|
||
fmt.Println(state.Running) | ||
|
||
connectionHost, err := scyllaContainer.ConnectionHost(ctx, 19042) // ShardAwareness port | ||
if err != nil { | ||
log.Printf("failed to get connection host: %s", err) | ||
return | ||
} | ||
|
||
runGoCQLExampleTest(connectionHost) | ||
|
||
// Output: | ||
// true | ||
} | ||
|
||
func ExampleRun_withConfigFile() { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. suggestion: as above, we probably want to use the |
||
ctx := context.Background() | ||
|
||
// runScyllaDBContainerWithConfigFile { | ||
scyllaContainer, err := scylladb.Run(ctx, | ||
"scylladb/scylla:6.2", | ||
scylladb.WithConfigFile(filepath.Join("testdata", "scylla.yaml")), | ||
) | ||
// } | ||
defer func() { | ||
if err := testcontainers.TerminateContainer(scyllaContainer); err != nil { | ||
log.Printf("failed to terminate container: %s", err) | ||
} | ||
}() | ||
if err != nil { | ||
log.Printf("failed to start container: %s", err) | ||
return | ||
} | ||
// } | ||
|
||
state, err := scyllaContainer.State(ctx) | ||
if err != nil { | ||
log.Printf("failed to get container state: %s", err) | ||
return | ||
} | ||
|
||
fmt.Println(state.Running) | ||
|
||
connectionHost, err := scyllaContainer.ConnectionHost(ctx, 9042) // Non ShardAwareness port | ||
if err != nil { | ||
log.Printf("failed to get connection host: %s", err) | ||
return | ||
} | ||
|
||
runGoCQLExampleTest(connectionHost) | ||
|
||
// Output: | ||
// true | ||
} | ||
|
||
func ExampleRun() { | ||
// runBaseScyllaDBContainer { | ||
ctx := context.Background() | ||
|
||
scyllaContainer, err := scylladb.Run(ctx, | ||
"scylladb/scylla:6.2", | ||
scylladb.WithShardAwareness(), | ||
) | ||
defer func() { | ||
if err := testcontainers.TerminateContainer(scyllaContainer); err != nil { | ||
log.Printf("failed to terminate container: %s", err) | ||
} | ||
}() | ||
if err != nil { | ||
log.Printf("failed to start container: %s", err) | ||
return | ||
} | ||
// } | ||
|
||
state, err := scyllaContainer.State(ctx) | ||
if err != nil { | ||
log.Printf("failed to get container state: %s", err) | ||
return | ||
} | ||
|
||
fmt.Println(state.Running) | ||
|
||
// BaseConnectionHost { | ||
connectionHost, err := scyllaContainer.ConnectionHost(ctx, 9042) | ||
// } | ||
if err != nil { | ||
log.Printf("failed to get connection host: %s", err) | ||
return | ||
} | ||
runGoCQLExampleTest(connectionHost) | ||
|
||
// Output: | ||
// true | ||
} | ||
|
||
func runGoCQLExampleTest(connectionHost string) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. suggestion: let's make this function to return the error, and fail the examples calling it if non nil. |
||
cluster := gocql.NewCluster(connectionHost) | ||
session, err := cluster.CreateSession() | ||
if err != nil { | ||
log.Printf("failed to create session: %s", err) | ||
} | ||
defer session.Close() | ||
|
||
var driver string | ||
err = session.Query("SELECT driver_name FROM system.clients").Scan(&driver) | ||
if err != nil { | ||
log.Printf("failed to query: %s", err) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: typo