markdowns done

.markdownlint.yml

@@ -0,0 +1,4 @@
+---
+default: true
+MD013:
+  line_length: 120

README.md

@@ -1,11 +1,15 @@
+# superset-cluster
+
 ![Tests](https://github.com/szachovy/superset-cluster/actions/workflows/tests.yml/badge.svg)
 ![Style](https://github.com/szachovy/superset-cluster/actions/workflows/style.yml/badge.svg)
 
-# superset-cluster
-
 Resilent Business Intelligence.
 
-* Survives multiple node failures with recovery mechanisms
+* Survives multiple node failures, recovery mechanisms are integrated.
+* No need for complex cluster management (e.g., Kubernetes).
+* Security by default, data in transit encryption.
+* Maintenance without downtime.
+* Implements caching and parallel job execution for enhanced performance.
 
 ![Demo](docs/demo.gif)
 
@@ -16,16 +20,20 @@ Resilent Business Intelligence.
 ### Hosts specification
 
 * Images are built and tested specifically for `Ubuntu 22.04 x86_64` Linux platforms shipped with `Python 3.10.12`.
-* Both `ssh` and `docker` services on the nodes must be enabled by default.
-* Nodes must be able to resolve DNS names between each other and be able to freely comunicate between each other in the internal network, only `443` port in the management nodes should be exposed.
-* The user's host must be able to `ssh` to each of the nodes passwordlessly.
+* Both `ssh` and `docker` services must be enabled by default on the nodes.
+* Nodes must be able to resolve DNS names among themselves and communicate freely over the internal network,
+  only port `443` should be exposed on the management nodes.
+* The user's host must be able to connect to each of the nodes via SSH passwordlessly.
 * Ability to read/write to the `/opt` directory on the nodes.
-* There should be at least one available and running network interface capable of sending and receiving packets between the user's host and management nodes via IPv4, IPv6 on this interface should be disabled or made non-routable default.
+* At least one available and running network interface must be capable of sending and receiving packets
+  between the user's host and management nodes via IPv4, IPv6 should be disabled or configured to be non-routable by default.
 
 ### Installed software
 
 The following software needs to be installed on the user's host:
-...
+
+* `python v3.10.12` with the following third party packages:
+  * `paramiko v3.5.0`
 
 The following software needs to be installed on the external nodes:
 
@@ -36,12 +44,15 @@ The following software needs to be installed on the external nodes:
 * `docker-ce v5:26.1.0-1~ubuntu.22.04~jammy`
 * `docker-ce-cli v5:26.1.0-1~ubuntu.22.04~jammy`
 * `openssh-server v1:8.9p1-3ubuntu0.10`
-
-...python packages... docker paramiko PYTHON VERSION IMPORTANT DUE TO MAGIC NUMBER
+* `Python v3.10.12` with the following third party packages:
+  * `docker v7.1.0`
 
 ## Usage
 
-Having two management nodes setup and three mysql nodes setup, run `./superset-cluster` with virtual ip address set to connect to on the specified network interface with a mask on the default gateway to be used as an example:
+To set up superset cluster with two management nodes and three MySQL nodes,
+run `./superset-cluster` with the virtual IP address configured for the
+specified available network interface. Use the following command as an example,
+including the appropriate subnet mask for the default gateway:
 
 ```bash
 ./superset-cluster \
@@ -52,21 +63,27 @@ Having two management nodes setup and three mysql nodes setup, run `./superset-c
   --virtual-network-mask 24
 ```
 
-Follow `./superset-cluster --help` for more information.
+For more information, run `./superset-cluster --help`.
+
+Navigate to `https://<virtual-ip-address>` in your web browser. Use the default credentials to log in:
 
-Next, navigate to https://192.168.1.100 in your web browser with the IP address provided in the installation to the default credentials username: superset, password: cluster
-Remember to change default credentials after successful log in the Settings > Info > Reset my password
+* Username: `superset`
+* Password: `cluster`
+
+Remember to change the default credentials after successfully logging in by going to _Settings > Info > Reset My Password_.
 
 ## Development
 
 For development purposes you can setup Terraform testing infrastructure being in `./tests/setup`, run:
 
+Set up the Terraform testing infrastructure for development, by navigating to the `./tests/setup` directory and running:
+
 ```bash
 terraform init
 terraform apply
 ```
 
-Consequently, run [Usage](#usage) command against this infrastructure. For default parameters it would be:
+Once the infrastructure is up and running, you can use the [Usage](#usage) command against it:
 
 ```bash
 ./superset-cluster \
@@ -77,19 +94,21 @@ Consequently, run [Usage](#usage) command against this infrastructure. For defau
   --virtual-network-mask 16
 ```
 
-Please refer to [the testing guide](tests/TESTING.md) for more information.
+For more detailed information, please refer to [the testing guide](tests/TESTING.md).
 
 ## License
 
 [Apache v2.0](LICENSE)
 
 ## Contributing
 
-If you notice anything missing, spot a bug, or have an enhancement proposal, feel free to open an issue with the appropriate label. Pull requests are welcome. Please ensure that the tests are updated as necessary.
+If you notice anything missing, spot a bug, or have an enhancement proposal,
+feel free to open an issue with the appropriate label.
+Pull requests are welcome. Please ensure that the tests are updated as necessary.
 
 ## Personal contact information
 
-In case of any inquiries, please write to email: wjmaj98@gmail.com
+In case of any inquiries, please write an email to: _wjmaj98@gmail.com_
 
 ## Additional resources
 

src/container.py

@@ -255,7 +255,7 @@ def run(self) -> None:
                 )
         # temporary
         import subprocess
-        command = 'docker login ghcr.io -u szachovy -p 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'
+        command = 'docker login ghcr.io -u szachovy -p ...'
         subprocess.run(command, shell=True, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
         print(self.wait_until_healthy(MySQLServer(self.client, str(self.container))))  # type: ignore[arg-type]
 
@@ -341,7 +341,7 @@ def run(self) -> None:
 
         # temporary
         import subprocess
-        command = 'docker login ghcr.io -u szachovy -p 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'
+        command = 'docker login ghcr.io -u szachovy -p ...'
         subprocess.run(command, shell=True, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
 
         return print(
@@ -464,7 +464,7 @@ def run(self) -> None:
 
         #temporary
         import subprocess
-        command = 'docker login ghcr.io -u szachovy -p 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'
+        command = 'docker login ghcr.io -u szachovy -p ...'
         subprocess.run(command, shell=True, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
         self.container = "redis"
         print(

src/crypto.py

@@ -50,6 +50,7 @@
 import os
 import random
 import string
+
 import cryptography
 import cryptography.hazmat.backends
 import cryptography.hazmat.primitives.hashes

tests/TESTING.md

@@ -17,26 +17,32 @@ Testing consist of two modules:
   * `<node-prefix>-2` is set as the primary mysql node
   * `<node-prefix>-3` is set as the secondary mysql node
   * `<node-prefix>-4` is set as the secondary mysql node
-* Each of the nodes have the same project's sources copied
 
 ### `testsuite`
 
 Ansible playbook running testsuite against applied terraform infrastructure, that include:
-* [Sanity](testsuite/roles/testing/tasks/sanity.yml) tests: Conditional checks if the [requirements](../README.md#requirements) of generated infrastructure are meet.
-* [System](testsuite/roles/testing/tasks/system.yml) tests: Simulated end-to-end deployment process of `superset-cluster` against terraform generated infrastructure.
-* [Functional](testsuite/roles/testing/tasks/functional.yml) tests: Verifying end-user primary use cases in pipelines of superset and management node component functions
+
+* [Sanity](testsuite/roles/testing/tasks/sanity.yml) tests:
+  Conditional checks if the [requirements](../README.md#requirements) of generated infrastructure are meet.
+* [System](testsuite/roles/testing/tasks/system.yml) tests:
+  Simulated end-to-end deployment process of `superset-cluster` against terraform generated infrastructure.
+* [Functional](testsuite/roles/testing/tasks/functional.yml) tests:
+  Verifying end-user primary use cases in pipelines of superset and management node component functions
 
 ## Running tests
 
 ### Required software
 
-Testing host is the `ubuntu:22.04` runner with 1 core of AMD EPYC 7773X x86_64 CPU and 8GiB of RAM with the [software installed](../README.md#installed-software) meeting the [required criteria](../README.md/#hosts-specification) with the following software installed for testing:
+Testing host is the `ubuntu:22.04` runner with 1 core of AMD EPYC 7773X x86_64 CPU and 8GiB of RAM
+with the [software installed](../README.md#installed-software)
+meeting the [required criteria](../README.md/#hosts-specification)
+with the following software installed for testing:
 
 * `terraform v1.0.10`
 * `python v3.10.12` with the following third party packages:
   * `ansible v9.5.1`
 
-Optionally, the following software is installed for doing [style checks](../.github/workflows/style.yml), and is present in the [automated testing](https://github.com/szachovy/superset-cluster/actions):
+Optionally, the following software is installed for doing [style checks](https://github.com/szachovy/superset-cluster/actions):
 
 * `pylint v3.2.4`
 * `mypy v1.10.1 (compiled)`
@@ -48,19 +54,19 @@ Optionally, the following software is installed for doing [style checks](../.git
 * `tflint v0.51.1`
 * `yamllint v1.35.1`
 
-For functional testing the [required](./testsuite/roles/testing/files/requirements.txt) python packages are installed on the deployed containers.
-Testing does not require any additional modules unpresent in the terraform initiated infrastructure, current versions of deployed software has been described in the project's [`README.md`](../README.md/#installed-software).
-
 ### Usage
 
-Being in the `setup` catalog run `terraform` commands to initialize infrastructure:
+Navigate to the `setup` directory and execute the following
+Terraform commands to initialize the infrastructure:
 
 ```bash
 terraform init
 terraform apply
 ```
 
-After successful run, and 5 docker containers appeared with a healthy state, run ansible playbook while being in the `testsuite` catalog to start tests:
+Once the commands have successfully completed,
+switch to the testsuite directory and run the Ansible playbook
+to start the tests:
 
 ```bash
 ansible-playbook --inventory inventory.yml deploy.yml
@@ -77,17 +83,20 @@ ansible-playbook --inventory inventory.yml deploy.yml
 | `node_version`  | Version of the container nodes deployed | "1.0"           |
 | `subnet`        | Network subnet for the container nodes  | "172.18.0.0/16" |
 
-### Ansible variables
+### Ansible Variables
 
-Ansible group variables is a dynamically modified file consisting of ...
+Ansible group variables are dynamically defined after terraform setup within a file. The following variables are used:
 
-virtual_network_interface: eth0
-virtual_ip_address: "172.18.0.10"
-virtual_network_mask: "255.255.0.0"
-node_prefix: "node"
+| Variable Name                | Description                                 | Default Value     |
+|------------------------------|---------------------------------------------|-------------------|
+| `virtual_network_interface`  | Network interface for the virtual network   | "eth0"            |
+| `virtual_ip_address`         | Virtual IP address assigned to the nodes    | "172.18.0.10"     |
+| `virtual_network_mask`       | Network mask for the virtual network        | "255.255.0.0"     |
+| `node_prefix`                | Prefix for the container nodes              | "node"            |
 
 ### Additional resources
 
 * [Terraform Docker provider](https://registry.terraform.io/providers/kreuzwerker/docker/latest/docs)
 * [Ansible Community Docker](https://docs.ansible.com/ansible/latest/collections/community/docker/index.html)
 * [Superset API](https://superset.apache.org/docs/api/#api)
+* [The MySQL Shell](https://dev.mysql.com/doc/mysql-shell/8.0/en/mysqlsh.html)

tests/testsuite/roles/testing/files/requirements.txt

@@ -1,3 +1 @@
 docker==7.1.0
-paramiko==3.5.0
-requests==2.32.3

tests/testsuite/roles/testing/tasks/sanity.yml

@@ -32,7 +32,7 @@
 - name: "Check out internal nodes settings"
   community.docker.docker_container_exec:
     container: "{{ container.value['Name'] }}"
-    command: "python3 -c 
+    command: "python3 -c
       'import introspect;
       introspect.ContainerChecks(
         node_prefix=\"{{ node_prefix }}\",
@@ -44,4 +44,4 @@
   loop_control:
     loop_var: "container"
   register: "introspect_command"
-  changed_when: "introspect_command.rc != 0"
+  changed_when: "introspect_command.rc != 0"