diff --git a/README.md b/README.md index 7ca0ce3..e1346f3 100644 --- a/README.md +++ b/README.md @@ -9,27 +9,1509 @@ This is the simple educational project prepared to support my presentation durin | :---: | :---: | :---: | :---: | | ![image](https://user-images.githubusercontent.com/36886649/135843518-9d4b2ec1-32dc-4226-a63c-b173d9b0706e.png) | ![image](https://user-images.githubusercontent.com/36886649/135534953-338af09d-d2c6-43ee-9407-137253cc4e13.png) | image | image +### Table of contents + +- [Requirements](#Requirements) +- [Docker and Docker Compose upgrade](#Docker-and-Docker-Compose-upgrade) +- [Project setup](#Project-setup) + - [Quick setup (without Docker cleanup)](#quick-setup-without-docker-cleanup) + - [Step by step setup](#Step-by-step-setup) + - [Clone repository and create directories](#Clone-repository-and-create-directories) + - [Introduction to docker and docker-compose](#Introduction-to-docker-and-docker-compose) + - [Docker cleanup (optional)](#docker-cleanup-optional) + - [Containers cleanup](#Containers-cleanup) + - [Networks cleanup](#Networks-cleanup) + - [Volumes cleanup](#Volumes-cleanup) + - [Development time cleanup](#Development-time-cleanup) + - [Docker compose project setup](#Docker-compose-project-setup) + - [Start services](#Start-services) + - [Install composer dependencies](#Install-composer-dependencies) + - [Elasticsearch index creation and population](#Elasticsearch-index-creation-and-population) + - [Make scripts executable](#Make-scripts-executable) + - [Quick setup with init.sh](#quick-setup-with-initsh) + - [Step by step setup with request.sh](#Step-by-step-setup-with-request.sh) + - [Create index](#Create-index) + - [Populate index with data](#Populate-index-with-data) + - [Get count of items](#Get-count-of-items) + - [Search](#Search) + - [Delete index](#Delete-index) + - [Step by step other methods](#Step-by-step-other-methods) +- [How to play with it?](#how-to-play-with-it) + - [Docker](#Docker) + - [Run all services](#Run-all-services) + - [Release the shell lock](#Release-the-shell-lock) + - [Run all services without shell locking](#Run-all-services-without-shell-locking) + - [Run a specific profile](#Run-a-specific-profile) + - [Stop services](#Stop-services) + - [Remove service containers](#Remove-service-containers) + - [Stop & remove service containers with their network](#stop--remove-service-containers-with-their-network) + - [Run, Stop, Remove](#run-stop-remove) + - [Build images](#Build-images) + - [Composer](#Composer) + - [Monitoring](#Monitoring) + - [Reinstallation](#Reinstallation) + - [Elasticsearch](#Elasticsearch) + - [Reinitialize](#Reinitialize) + - [Elasticsearch code](#Elasticsearch-code) +- [Uninstallation](#Uninstallation) +- [Credits](#Credits) +- [Copyrights](#Copyrights) + + [^TOC^](#Table-of-contents) + ## Requirements -- PHP 8.0.9 -- Symfony CLI (https://symfony.com/download) -- Elasticsearch 7.16.0 running on `localhost:9200` -If you need to change the Elasticsearch host the application uses, [it's defined in the `ApiClient` class as a constant](https://github.com/lrynek/painless-car-rental/blob/b4a8431ffd73c7417b00d6428ef491c91b45960f/src/Elasticsearch/Service/ApiClient.php#L14) (normally worth passing it from `.env` params file 😉 ) +- docker (tested on v20.10.13) +- docker-compose (tested on v2.3.3) +- linux (tested on Ubuntu 18.04) + + [^TOC^](#Table-of-contents) + +## Docker and Docker Compose upgrade + +If you use the `docker` and/or `docker-compose` provided by your system vendor - you installed it on Ubuntu for eg. by `apt-get install docker` or `apt-get install docker-compose` then probably at least your installation of the `docker-compose` has to low version to run this application. + +You may need to also to update the version of the `docker` but for the Ubuntu it can be done by its GUI [Software Updater](https://en.wikipedia.org/wiki/Ubuntu_Software_Updater). + + + +You may check your current version of the `docker-compose` by running: + +```sh +docker-compose --version +``` + +If it is `1.x` then you need to upgrade it. + + + +**Ubuntu `docker-compose` most recent version installation**: + +Get info about available version to download: + +```sh +VERSION="$(curl --silent 'https://api.github.com/repos/docker/compose/releases/latest' | grep -Po '"tag_name": "\K.*\d')" && echo "Available version: $VERSION" +``` + +example terminal output: + +```sh +Available version: v2.3.3 +``` + +if you don't see any version here discontinue and do not follow other steps but try on your own to upgrade the `docker-compose`. + + + +Check if you have `docker-compose` installed and where, run: + +```sh +DESTINATION="$(find / -iname 'docker-compose' -type f -executable 2> /dev/null || true)" && +{ [ -z ${DESTINATION:-} ] && echo "DESTINATION must be set manually"; } || echo "DESTINATION is set to: $DESTINATION" +``` + +example terminal output: + +``` +/usr/bin/docker-compose +``` + +or "DESTINATION must be set manually" if you haven't installed `docker-compose`, then variable `DESTINATION` must be later set by you manually (continue with the steps as below). + + + +Uninstall current version only if DESTINATION is set: + +```sh +if [ -z "${DESTINATION:-}" ]; then + echo "No need for uninstalling, docker-compose not found" +else + echo "Uninstalling docker-compose" + sudo apt-get remove docker-compose +fi +``` + + + +in case you have seen: + +> DESTINATION must be set manually + +now is the time to set the destination to the path you wish, for example run: + +```sh +DESTINATION=/usr/bin/docker-compose +``` + + + +Download the `docker-compose` file and make it executable: + +```sh +echo "Downloading file to: ${DESTINATION:?}" +sudo curl -L "https://github.com/docker/compose/releases/download/${VERSION:?}/docker-compose-$(uname -s)-$(uname -m)" -o "${DESTINATION:?}" + +echo "Changing permissions to rx" +sudo chmod a=rx "${DESTINATION:?}" +``` + + + +Now you should be able to run `docker-compose` and check its new version, run: + +``` +docker-compose --version +``` + +example terminal output: + +``` +Docker Compose version v2.3.3 +``` + + + +If by this time you don't have installed new version of the `docker-compose` then you may install again the previous version of the `docker-compose` by running: + +```sh +sudo apt-get install docker-compose +``` + +from the official repository or search for some alternative methods of installing more recent version than the one you currently have. + + + +## Project setup + +This process is available in two flavors: + +1. Quick setup (if you can't wait to play) +2. Step by step approach (has additional explanations) + +Either way the end result that can be checked by the command: + +```sh +docker ps -a --format '{{.Image}} {{.Status}} {{.Ports}}' +``` + + should be: + +| IMAGE | STATUS | PORTS | +| ------------------------------ | ---------------------- | --------------------------------------------------- | +| carrental_nginx:1.17.8 | Up 9 minutes (healthy) | 0.0.0.0:9090->80/tcp, :::9090->80/tcp | +| carrental_composer:2.2.7 | Up 9 minutes | | +| carrental_php:8.1.3-fpm-buster | Up 9 minutes | 9000/tcp | +| carrental_elasticsearch:7.17.1 | Up 9 minutes (healthy) | 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp | + + [^TOC^](#Table-of-contents) + +## Quick setup (without Docker cleanup) + +1. Clone this repository: + + ```sh + git clone 'https://github.com/212223/painless-car-rental.git' + ``` + +3. Change directory: + + ```sh + cd painless-car-rental/docker/service + ``` + +3. Make `setup.sh` script executable && run `script.sh`: + + ```sh + sudo chmod ug+x ./setup.sh && ./setup.sh + ``` + + `./setup.sh` may ask you for the sudo password because it is required for changing permissions to the directories it creates. + + Feel free to check `./setup.sh` content if you are wary, run: + + ```sh + cat ./setup.sh + ``` + + + + **Do not** run `./setup.sh` as a sudo user. Wrong, do **not** do this: + + ```sh + sudo ./setup.sh + ``` + + because it needs to run as your regular user in order to use your user id and group id and make correct ownership of directories. + + + + If you see an error like this: + + > Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "...": dial unix /var/run/docker.sock: connect: permission denied + + This means that your user don't have permissions to run docker. + + Add your user to the `docker` group, run: + + ```sh + sudo usermod -a -G docker $USER + ``` + + Reload docker: + + ```sh + sudo systemctl restart docker + ``` + + Relog your user: + + ```sh + su - $USER + ``` + + Check if your user is added to the `docker` group, run: + + ```sh + groups + ``` + + example terminal output: + + ``` + test sudo docker + ``` + + Once you see `docker` group then run setup script again (without sudo): + + ```sh + ./setup.sh + ``` + + + + If you see an error: + + > * error listing credentials - err: exit status 1, out: `Error from list function in secretservice_linux.c likely due to error in secretservice library` + + Install these packages: + + ```sh + sudo apt-get install -y pass gnupg2 + ``` + + + +4. Open your browser and go to: + http://localhost:9090 + +4. Read the section [How to play with it?](#how-to-play-with-it) + +5. Read about the project structure in [./docker/service/README.md](./docker/service/README.md) + + [^TOC^](#Table-of-contents) + +## Step by step setup + + + +#### Clone repository and create directories + +Clone this repository && change dir to it: + +```sh +git clone 'https://github.com/212223/painless-car-rental.git' && cd painless-car-rental/docker/service +``` + + + +Export your current shell user id and its group id, run: + +```sh +export uid=$(id -u) +export gid=$(id -g) +``` + +to have all `docker-compose` commands run as your current user and own files. + +If not then they will be run as user:group = 1000:1000 or 1000:0 for Elasticsearch. + + + +Own all directories and files: + +```sh +sudo chown -R $uid:$gid ../../ +``` + + + +Add execution for the current user and its group to init scripts + +```sh +sudo chmod -v ug+x,o-wx \ +../../app/default/.elasticsearch-http-requests/request.sh \ +../../app/default/.elasticsearch-http-requests/init.sh +``` + + + +If the docker attempts to mount at HOST a directory that does not exists it will be created by the docker demon. + +The problem is that directory will be created by docker that runs as a root therefore it will be owned by the user id = 0. + +That directory will be not accessible for all of the processes that run within containers with different user id. + +This application runs all containers with the same user id and group id as the user who started them. + +Because of that it is possible to freely use PHP composer for example without any sudo as well deleting all files without any sudo. + +Running containers as a regular user also enhances security and makes possible to use file access permissions different for user, group and others (which makes no difference if the user is root). + +In order to have access permissions from within containers that run processes as a regular user all of the directories that are used as host's mounting points must be created upfront and have desired permissions before running the docker commands. + + + +Create directory for Nginx: + +```sh +mkdir -p \ +./nginx-v1/image/files/var/log/nginx \ +./nginx-v1/image/files/usr/share/nginx/html/default && \ +\ +sudo chown -R $uid:$gid \ +./nginx-v1/image/files/var/log/nginx \ +./nginx-v1/image/files/usr/share/nginx/html/default && \ +\ +sudo chmod -v -R ug=rwX,o=rX \ +./nginx-v1/image/files/var/log/nginx \ +./nginx-v1/image/files/usr/share/nginx/html/default +``` + + + +Create directory for Elasticsearch + +```sh +mkdir -p \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/config \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/data \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/logs \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/plugins && \ +\ +sudo chown -R $uid:0 \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/config \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/data \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/logs \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/plugins && \ +\ +sudo chmod -v -R ug=rwX,o=rX \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/config \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/data \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/logs \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/plugins +``` + + + [^TOC^](#Table-of-contents) + +## Introduction to docker and docker-compose + +Feel free to use `--help` option before running any `docker` or `docker-compose` command to know what you are doing. + +run: + +```sh +docker --help +``` + +or: + +```sh +docker-compose --help +``` + +to get information about `docker` or `docker-compose` commands. + + + +You may get also use `--help` for the info related to a specific sub-command: + +```sh +docker ps --help +``` + +or: + +```sh +docker-compose ps --help +``` + + + +The command `docker` should work without any problem anywhere. + +However using `docker-compose` commands usually requires your shell to be inside the directory where the `docker-compose.yaml` file is ([./docker/service](./docker/service) for this project). + + + +Note that if you use sub commands then command-line options switches are available (and potentially different) for both commands for eg. `docker-compose` and its subcommand `up` + +```sh +docker-compose --help +``` + +will display different option switches than + +```sh +docker-compose up --help +``` + +keep that in mind and place option switches right after the command they relate to. for example: + +wrong: + +```sh +docker-compose up --build --profile dev +``` + +terminal output: + +> unknown flag: --profile + +subcommand `up` of the `docker-compose` doesn't have the `--profile` switch therefore `--build` option must be placed right after the `docker-compose` command. + +correct: + +```sh +docker-compose --profile dev up --build +``` + + [^TOC^](#Table-of-contents) + +## Docker cleanup (optional) + +Before running this application you may want to do the Docker cleanup and remove all of the containers and networks. + +Make sure you that you do not have any: + +- container +- docker network +- dandling images +- dandling build cache + +that is important (ie a container(s) handling an incoming traffic or saving a backup data) before proceeding with the steps as described in this section. + +**Running** the **Docker cleanup** section commands **is optional** but should give you a clean starting point and lack of confusion by having only the containers and networks that are related to this project. + + [^TOC^](#Table-of-contents) + +#### Containers cleanup + +To see if you have any containers created run: + +```sh +docker ps -a +``` + +During the development time you may stop all containers, run: + +```sh +docker-compose stop +``` + +Then you may remove all of them, run: + +```sh +docker-compose down +``` + +But then still some containers may be presented by `docker ps -a` command. If you want to remove all of the containers, even those that are not related to this application project then run: + +```sh +docker rm $(docker ps -aq) +``` + +In case of any complains from docker you may add flag `--force` but read and understand what the docker is complaining on and decide if you really want to force docker to remove all of the containers: + +```sh +docker rm --force $(docker ps -aq) +``` + +to see effect run again: + +```sh +docker ps -a +``` + +terminal output: + +``` +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +``` + +should show no containers (despite their state: created, stopped, running, exited etc.) + + [^TOC^](#Table-of-contents) + +#### Networks cleanup + +You may also remove all of the docker networks that you don't need any longer including networks used by the docker containers you've just removed. + +to list all of the networks run: + +```sh +docker network ls +``` + +to remove all of the user networks: + +```sh +docker network rm $(docker network ls -q --filter 'type=custom') +``` + +run: + +```sh +docker network ls +``` + +terminal output: + +``` +NETWORK ID NAME DRIVER SCOPE +5d5de2f7ccd4 bridge bridge local +b75e841984e3 host host local +efff0416acdb none null local +``` + +should show that there are no other networks than the default ones. + + [^TOC^](#Table-of-contents) + +#### Volumes cleanup + +This application does not make any use of volumes (besides the ones that are defined by the images' Dockerfiles). Therefore it is not needed nor recommended to do a cleanup of them for the purpose of running this application. + + [^TOC^](#Table-of-contents) + +#### Development time cleanup + +At first time of application run it is not necessary to follow this instruction - you may skip it. + +Once you start working with this application and modify it, stop, remove, build and run again you may find that you have some containers or networks that are not used but still present when you issue the `docker ps -a` or `docker network ls` command. + +In such case, you may find useful + +Following steps from [Run, Stop, Remove](#run-stop-remove) to run and do all cleanup after shutting the application down. + +or using this command: + +```sh +docker system prune +``` + +that outputs in the terminal: + +``` +WARNING! This will remove: + - all stopped containers + - all networks not used by at least one container + - all dangling images + - all dangling build cache + +Are you sure you want to continue? [y/N] +``` + +The terminal output is pretty self explanatory however if you would like to check the documentation: + +- [docker system prune](https://docs.docker.com/engine/reference/commandline/system_prune/) +- [docker container prune](https://docs.docker.com/engine/reference/commandline/container_prune/) +- [docker network prune](https://docs.docker.com/engine/reference/commandline/network_prune/) + + [^TOC^](#Table-of-contents) + +## Docker compose project setup + +#### Start services + +Do not run any of the commands with `sudo` unless it is clearly used. + +If you run these commands with `sudo` then the user id will be 0 as well as the group id. That will affect all of the containers that are built and used with the same user id and group id as the user that issued the commands to built and run them. + +You will have no benefit of running the commands with `sudo` but only will degrade security and make setting access permissions for user, group and others pointless. + +If you have any problems running docker commands without `sudo` then you need to add your user to the docker group, read about that in the [Quick setup (without Docker cleanup)](#quick-setup-without-docker-cleanup) section. + +run: + +```sh +docker-compose --profile dev up --detach --build --force-recreate +``` + +to get the `dev` profile running and wait approx 40 sec to finish. + +Once done and shell is accepting new commands, terminal output: + +``` +[+] Running 4/4 + ⠿ Container carrental-php-composer-v2-1 Started 2.2s + ⠿ Container carrental-elasticsearch-v7-1 Healthy 27.1s + ⠿ Container carrental-php-fpm-v8-1 Started 3.0s + ⠿ Container carrental-nginx-v1-1 Started 28.0s +``` + + + +run: + +```sh +docker ps -a --format '{{.Image}}\t{{.Status}}\t{{.Names}}' +``` + +to see all containers created, terminal output: + +``` +carrental_nginx:1.17.8 Up 7 seconds (healthy) carrental-nginx-v1-1 +carrental_php:2.2.7-composer Up 31 seconds carrental-php-composer-v2-1 +carrental_php:8.1.3-fpm-buster Up 30 seconds carrental-php-fpm-v8-1 +carrental_elasticsearch:7.17.1 Up 29 seconds (healthy) carrental-elasticsearch-v7-1 +``` + +with both `carrental-nginx-v1-1` and `carrental-elasticsearch-v7-1` status **(healthy)** + +for the full info you may run: + +```sh +docker ps -a +``` + + + +or open a new terminal window and run: + +```sh +docker stats +``` + +that should show containers and their resources usage. + +If you have opened the second terminal window, go back to the first terminal window and continue with instructions: + + [^TOC^](#Table-of-contents) + +#### Install composer dependencies + +```sh +docker-compose exec php-composer-v2 composer install +``` + +terminal output: + +``` +Generating optimized autoload files +38 packages you are using are looking for funding. +Use the `composer fund` command to find out more! + +Run composer recipes at any time to see the status of your Symfony recipes. + +Executing script cache:clear [OK] +Executing script assets:install public [OK] +``` + +At this moment the PHP application is set but there is still missing: + + [^TOC^](#Table-of-contents) + +## Elasticsearch index creation and population + +#### Make scripts executable + +**Make `request.sh` script executable** + +```sh +docker-compose exec php-composer-v2 /bin/sh -c "chmod -cv ug+x /app/default/.elasticsearch-http-requests/request.sh" +``` + +terminal output: + +``` +... 0754 (rwxr-xr--) +``` + +so your user and his group are able to execute that script. + + [^TOC^](#Table-of-contents) + +**Make `init.sh` script executable** + +```sh +docker-compose exec php-composer-v2 /bin/sh -c "chmod -cv ug+x /app/default/.elasticsearch-http-requests/init.sh" +``` + +terminal output: + +``` +... 0754 (rwxr-xr--) +``` + +so your user and his group are able to execute that script. + + [^TOC^](#Table-of-contents) + +**There are at least three ways to create Elasticsearch's index and populate it** + +- Quick setup with `init.sh` - just gets you the DB set. +- Step by step with `request.sh` - provides more fun. +- Step by step other methods (require manual edit of file(s)) + + [^TOC^](#Table-of-contents) + +#### Quick setup with init.sh + +```sh +docker-compose exec php-composer-v2 /bin/sh -c "/app/default/.elasticsearch-http-requests/init.sh" +``` + +Terminal output: + +``` +... + +Performing cURL request: +curl -sSL -X GET 'http://elasticsearch-v7:9200/cars/_count' +{"count":39,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}} +Success. +Done! +``` + +That's it. Open your bowser and visit http://localhost:9090/ + + [^TOC^](#Table-of-contents) + +#### Step by step setup with request.sh + +##### Create index + +```sh +docker-compose exec php-composer-v2 /bin/sh -c "/app/default/.elasticsearch-http-requests/request.sh /app/default/.elasticsearch-http-requests/Index/Create.http" +``` + +terminal output: + +``` +Performing cURL request: +... +{"acknowledged":true,"shards_acknowledged":true,"index":"cars"} +Success. +``` + + [^TOC^](#Table-of-contents) + +##### Populate index with data + +``` +docker-compose exec php-composer-v2 /bin/sh -c "/app/default/.elasticsearch-http-requests/request.sh /app/default/.elasticsearch-http-requests/Index/Bulk.http" +``` + +terminal output: + +``` +Performing cURL request: +curl -sSL -X POST -H 'Content-Type: application/x-ndjson' --data-raw \ +'{"update":{"_index":"cars","_type":"_doc","_id":"1"}} +{"doc":{"producer":"Ferrari","model":"SF90","picture":"ferrari-sf90.jpg","production_year":2021,"colors":["red"],"service_ids":[2,4,6,8]},"doc_as_upsert":true} + +... + +{"update":{"_index":"cars","_type":"_doc","_id":"39","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":38,"_primary_term":1,"status":201}}]} +Success. +``` + + [^TOC^](#Table-of-contents) + +##### Get count of items + +```sh +docker-compose exec php-composer-v2 /bin/sh -c "/app/default/.elasticsearch-http-requests/request.sh /app/default/.elasticsearch-http-requests/Index/Count.http" +``` + +terminal output: + +``` +Performing cURL request: +curl -sSL -X GET 'http://elasticsearch-v7:9200/cars/_count' +{"count":39,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}} +Success. +``` + + [^TOC^](#Table-of-contents) + +##### Search + +```sh +docker-compose exec php-composer-v2 /bin/sh -c "/app/default/.elasticsearch-http-requests/request.sh /app/default/.elasticsearch-http-requests/Search/Phrase.http" +``` + +terminal output: + +``` +Performing cURL request: +curl -sSL -X POST -H 'Content-Type: application/json' --data-raw \ +'{ + "_source": true, + "explain": false, + +... + +{"producer":"McLaren","model":"720S","picture":"mclaren-720s.jpg","production_year":2018,"colors":["blue","orange","black"],"service_ids":[7,2,1,3]}}]}} +Success. +``` + +That's it. Open your bowser and visit http://localhost:9090/ and + +Read the section [How to play with it?](#how-to-play-with-it). + + [^TOC^](#Table-of-contents) + +##### Delete index + +```sh +docker-compose exec php-composer-v2 /bin/sh -c "/app/default/.elasticsearch-http-requests/request.sh /app/default/.elasticsearch-http-requests/Index/Delete.http" +``` + +terminal output: + +``` +Performing cURL request: +curl -sSL -X DELETE 'http://elasticsearch-v7:9200/cars' +{"acknowledged":true} +Success. +``` + +Terminal may print `FAIL!` in case there was no index to delete (you deleted it before, or did not create). + + [^TOC^](#Table-of-contents) + +#### Step by step other methods + +All HTTP requests can be executed either: +- from within [PhpStorm's built-in REST HTTP client](https://www.jetbrains.com/help/phpstorm/http-client-in-product-code-editor.html) (samples in [.elasticsearch-http-requests directory](https://github.com/lrynek/phpers-2021/blob/main/.elasticsearch-http-requests)) +- in [Insomnia REST HTTP client](https://insomnia.rest/) (import [insomnia.json file](https://github.com/lrynek/phpers-2021/blob/main/insomnia.json) with all the samples) + +You may receive an error from Elastisearch that complains about nested document. If so then add this query + +``` +?include_type_name=true +``` + +to your `PUT` request like to this one: `app/default/.elasticsearch-http-requests/Index/Create.http` + +first line: + +```http +PUT http://localhost:9200/cars +``` + +should become: + +```http +PUT http://localhost:9200/cars?include_type_name=true +``` + +Keep in mind that using `localhost` as a request's host name is only valid if you make that request from your machine (not from the composer docker container). + +Therefore if you want to do them manually but from a docker container you have two choices: + +- Make requests from within the Elasticsearch container because there the `localhost` points to the Elasticsearch's service internal [loopback](https://en.wikipedia.org/wiki/Loopback). +- Change the `localhost` in the request to `elasticsearch-v7` or to the Elasticsearch container name available under `docker ps` or Elasticsearch service network alias that is available by using the command: `docker inspect elasticsearch-container-name` + + + +Open your bowser and visit http://localhost:9090/ + + [^TOC^](#Table-of-contents) + +## How to play with it? + +Read about the project structure in [./docker/service/README.md](./docker/service/README.md) + +or continue with the content as below: + + [^TOC^](#Table-of-contents) + +## Docker + + [^TOC^](#Table-of-contents) + + + +Before doing any commands from this section make sure that your terminal's current directory is [./docker/service](./docker/service) + + + +#### Run all services + +To run this application you need to specify explicitly the profile you want to run. + +Profile `dev` runs all of the available services. To run the `dev` profile type: + +```sh +docker-compose --profile dev up +``` + +Note `--profile` option comes after the `docker-compose` command, not after the `up` subcommand. + +Your terminal shell will be locked and you will not be able to type into it but shell will print logs from any service container instead: + +``` +carrental-php-fpm-v8-1 | [17-Mar-2022 15:08:41] NOTICE: fpm is running, pid 1 +carrental-php-fpm-v8-1 | [17-Mar-2022 15:08:41] NOTICE: ready to handle connections +carrental-elasticsearch-v7-1 | {"type": "server", "timestamp": "2022-03-17T15:08:42,246Z" +``` + + [^TOC^](#Table-of-contents) + +##### Release the shell lock + +Use the keyboard combination of Ctrl+c. + +The Ctrl+c stops all of the containers created by `docker-compose` `up` + +``` +[+] Running 4/4 +⠿ Container carrental-nginx-v1-1 Stopped +⠿ Container carrental-elasticsearch-v7-1 Stopped +⠿ Container carrental-php-composer-v2-1 Stopped +⠿ Container carrental-php-fpm-v8-1 Stopped +``` + +and gives the shell control back to you. + + [^TOC^](#Table-of-contents) -In order to run the project, it is advisable to install an instance of latest stable version of Elasticsearch (it's 7.16.0 version at the moment of the presentation https://www.elastic.co/guide/en/elasticsearch/reference/7.16/index.html) +#### Run all services without shell locking -## Setup -1. Create `cars` index in Elasticsearch ([`Index/Create` HTTP request](https://github.com/lrynek/painless-car-rental/blob/main/.elasticsearch-http-requests/Index/Create.http)*) -2. Populate the index with sample cars data ([`Index/Bulk` HTTP request](https://github.com/lrynek/painless-car-rental/blob/main/.elasticsearch-http-requests/Index/Bulk.http)*) -3. Go to project's root directory in the terminal -4. Start Symfony server `symfony server:start --no-tls` -5. Go to http://127.0.0.1:8000/ +If you don't want `docker-compose` `up` to hold your shell you may use `--detach` or `-d` flag. ->(*) - all HTTP requests can be executed either: ->- from within [PhpStorm's built-in REST HTTP client](https://www.jetbrains.com/help/phpstorm/http-client-in-product-code-editor.html) (samples in [.elasticsearch-http-requests directory](https://github.com/lrynek/painless-car-rental/blob/main/.elasticsearch-http-requests)) ->- in [Insomnia REST HTTP client](https://insomnia.rest/) (import [insomnia.json file](https://github.com/lrynek/painless-car-rental/blob/main/insomnia.json) with all the samples) +```sh +docker-compose --profile dev up --detach +``` + +after starting all of the services: + +``` +[+] Running 6/6 + ⠿ Network carrental_backend Created 0.1s + ⠿ Network carrental_frontend Created 0.1s + ⠿ Container carrental-php-fpm-v8-1 Started 1.9s + ⠿ Container carrental-php-composer-v2-1 Started 2.9s + ⠿ Container carrental-elasticsearch-v7-1 Healthy 24.4s + ⠿ Container carrental-nginx-v1-1 Started 25.6s +``` + +the shell will be released and waiting for your input. + +However the downside is that you will not have the terminal output view with information coming from the services like it was printed without using the `--detach` flag. + +To solve that issue you run this command preferably in a new terminal window: + +```sh +docker-compose logs +``` + +that will provide you the same output as running the `docker-compose` `up` that locks the shell. + +When running `docker-compose` `up` with `--detach` key combination Ctrl+c will not stop and remove containers with their networks. + +To address that issue check commands from sections as below: + +- [Stop services](#Stop-services) (equivalent of Ctrl+c) +- [Remove service containers](#Remove-service-containers) +- [Stop & remove service containers with their network](#stop--remove-service-containers-with-their-network) + + [^TOC^](#Table-of-contents) + +#### Run a specific profile + +Services and profiles that they are assigned to are listed in [./docker/service/_profiles.yaml](./docker/service/_profiles.yaml) + + + +This section assumes you have created containers by steps from [Run all services](#Run-all-services) and stopped them as described in: [Release the shell lock](#Release-the-shell-lock). + +If you want to run a specific profile you know that you need to specify `--profile` option followed by the profile name. + +Let's run `cli` profile that has only the `php-composer-v2` service assigned to it, run: + +```sh +docker-compose up --profile cli +``` + +terminal output: + +``` +[+] Running 1/0 + ⠿ Container carrental-php-composer-v2-1 Created 0.0s +Attaching to carrental-elasticsearch-v7-1, carrental-nginx-v1-1, carrental-php-composer-v2-1, carrental-php-fpm-v8-1 + +... +``` + +Did you notice something strange? + +If not then open a new terminal window Ctrl+Alt+t and run: + +```sh +docker ps -a +``` + +Check how many containers are running, it should be there just one: `carrental-php-com +poser-v2-1` because that's the only service the profile `cli` has. + +It looks that by running `docker-compose --profile cli` command we end up having all of the containers running from the `dev` profile. Why is that? + +Notice terminal output of the `docker-compose --profile cli`, especially the line at the beginning: + +> [+] Running 1/0 +> ⠿ Container carrental-php-composer-v2-1 Created 0.0s + +That looks as expected, we wanted to create just one service that the `cli` profile consist of but besides that also something else has happened: + +> Attaching to carrental-elasticsearch-v7-1, carrental-nginx-v1-1, carrental-php-composer-v2-1, carrental-php-fpm-v8-1 + +Use the keyboard combination of Ctrl+c to stop all of the containers. + +Terminal output: + +``` +[+] Running 4/4 +⠿ Container carrental-nginx-v1-1 Stopped +⠿ Container carrental-elasticsearch-v7-1 Stopped +⠿ Container carrental-php-composer-v2-1 Stopped +⠿ Container carrental-php-fpm-v8-1 Stopped +``` + +In order to have running only `carrental-php-composer-v2-1` [Remove service containers](#Remove-service-containers) + +and run profile `cli` again: + +```sh +docker-compose up --profile cli +``` + +Terminal output: + +``` +[+] Running 2/2 + ⠿ Network carrental_backend Created 0.1s + ⠿ Container carrental-php-composer-v2-1 Created 0.1s +Attaching to carrental-php-composer-v2-1 +``` + +As you can see there is only one container created and attaching is done to it. No other containers were attached because they did not exist (they were removed). + + + +If you want always to run a specific profile then edit the [.env](./docker/service/.env) file and put there: + +``` +COMPOSE_PROFILES=dev +``` + +so you should be able to issue only the: + +```sh +docker-compose up +``` + +command to run the `dev` profile. + +All profiles that you may choose from are defined in the [_profiles.yaml](./docker/service/_profiles.yaml) file in the same directory where the docker-compose.yaml file is [./docker/service](./docker/service) + +Documentation: [Using profiles with Compose](https://docs.docker.com/compose/profiles/) + + [^TOC^](#Table-of-contents) + +#### Stop services + +It makes sense to use it only if you plan to restart these containers later. + +If your intention is to remove services then use command provided in [Remove service containers](#Remove-service-containers) that may stop containers as well before removing them or use [Stop & remove service containers with their network](#stop--remove-service-containers-with-their-network) that is able to do the same plus networks removal. + +If you want to only stop docker containers that were started by + +```sh +docker-compose up +``` + +without `--deatach` flag then you need to press in that terminal Ctrl+c. + +If the process of stopping containers takes too long then you may press Ctrl+c again to kill containers. Keep in mind that may result in the data loss. + +if you started containers with the`--deatach` flag then you need to issue a command + +```sh +docker-compose stop +``` + +Example terminal output: + +``` +[+] Running 4/4 +⠿ Container carrental-nginx-v1-1 Stopped +⠿ Container carrental-elasticsearch-v7-1 Stopped +⠿ Container carrental-php-composer-v2-1 Stopped +⠿ Container carrental-php-fpm-v8-1 Stopped +``` + + [^TOC^](#Table-of-contents) + +#### Remove service containers + +You may remove service containers that are already stopped or even running containers if `--force` flag added: + +```sh +docker-compose rm --force --stop +``` + +options: + +- `-f`, `--force` Don't ask to confirm removal +- `-s`, `--stop` Stop the containers, if required, before removing + +example terminal output: + +``` +[+] Running 4/4 + ⠿ Container carrental-php-composer-v2-1 Stopped 10.4s + ⠿ Container carrental-nginx-v1-1 Stopped 0.6s + ⠿ Container carrental-php-fpm-v8-1 Stopped 0.3s + ⠿ Container carrental-elasticsearch-v7-1 Stopped 0.8s + +Going to remove carrental-nginx-v1-1, carrental-elasticsearch-v7-1, carrental-php-fpm-v8-1, carrental-php-composer-v2-1 + +[+] Running 4/0 + ⠿ Container carrental-php-composer-v2-1 Removed 0.0s + ⠿ Container carrental-nginx-v1-1 Removed 0.0s + ⠿ Container carrental-elasticsearch-v7-1 Removed 0.0s + ⠿ Container carrental-php-fpm-v8-1 Removed 0.0s +``` + +If you want to remove also networks then look at: [Stop & remove service containers with their network](#stop--remove-service-containers-with-their-network) + + [^TOC^](#Table-of-contents) + +#### Stop & remove service containers with their network + +[Remove service containers](#Remove-service-containers) only removes containers but what if you need to get rid of all of the containers with their networks for eg. you reconfigured network and want to do `docker-compose` `up` with the new network configuration? + +You may use for that `docker-compose` `down` that is the opposite command to the `docker-compose` `up`, run: + +```sh +docker-compose down --remove-orphans +``` + +Options: + +- `--remove-orphans` Remove containers for services not defined in the Compose file. + +example terminal output: + +``` +[+] Running 6/6 + ⠿ Container carrental-php-composer-v2-1 Removed 10.5s + ⠿ Container carrental-elasticsearch-v7-1 Removed 1.0s + ⠿ Container carrental-php-fpm-v8-1 Removed 0.5s + ⠿ Container carrental-nginx-v1-1 Removed 1.0s + ⠿ Network carrental_backend Removed 0.3s + ⠿ Network carrental_frontend Removed 0.2s +``` + +Note that networks: + +``` + ⠿ Network carrental_backend Removed 0.3s + ⠿ Network carrental_frontend Removed 0.2s +``` + +were also removed in contrary to the [Remove service containers](#Remove-service-containers) that removes only services. + + [^TOC^](#Table-of-contents) + +#### Run, Stop, Remove + +Make sure you that you do not have any: + +- container +- docker network + +that is important (ie a container(s) handling an incoming traffic or saving a backup data) before proceeding with the steps as described in this section. + + + +When developing an application it is very often need to do a repetitive cycle of docker commands: + +1. Run all or just selected services +2. Shut everything down +3. Clean any leftovers +4. Build the images again +5. Repeat point 1 with updated configuration + + + +Below is a one-liner that covers **points 1~4** for the `dev` profile (all services). + +Run and cleanup this project: + +```sh +docker-compose --profile dev up --build --force-recreate; \ +docker-compose down --remove-orphans; +``` + +See description of commands below. + +Run and cleanup every project: + +```sh +docker-compose --profile dev up --build --force-recreate; \ +docker-compose down --remove-orphans; \ +containers="$(docker ps -aq)"; [ -z "$containters" ] || docker rm -f $containers; \ +networks="$(docker network ls -q --filter 'type=custom')"; [ -z "$networks" ] || docker network rm $networks; +``` + +Explanation: + +- `docker-compose --profile dev up --build --force-recreate` runs the `dev` profile (`--profile`) and builds (`--build`) all images always (`--force-recreate`). Blocks the terminal input (lack of `--deatch`) till a user presses Ctrl+c. + +- `docker-compose down --remove-orphans` shuts all containers down and removes them with their networks. + +- `containers="$(docker ps -aq)"; [ -z "$containters" ] || docker rm -f $containers;` removes all of the containers even these ones that are not related to the project. + +- `networks="$(docker network ls -q --filter 'type=custom')"; [ -z "$networks" ] || docker network rm $networks` removes all of the user created networks even these that are not related to this project. + + + +**Point 5**: To repeat that command use Ctrl+Up Arrow. Once it is displayed press Enter. + + [^TOC^](#Table-of-contents) + +#### Build images + +If you make any changes to the Dockerfiles you should use `--build` flag so the `docker-compose` `up` rebuilds your images before running them again. + +example: + +```sh +docker-compose --profile dev up --build +``` + +Sometimes even you have made changes to the Dockerfile or a service `.yaml` file of an image docker will use its old version, therefore `--force-recreate` is useful during the development phase: + +```sh +docker-compose --profile dev up --build --force-recreate +``` + + + +If you run your `docker-compose` `up` command with newly build images but still see that the old image is used check with the command: + +```sh +docker ps -a +``` + +name of the containers under the column `IMAGE`. + +If the column has a hash value like: `26d6a19a43ba` instead of a container name for eg: `carrental_composer:2.2.7` then something went wrong and the docker has not started with the newly created image but with an old one. + +In such case you may need to: + +- Check if a new Dockerfile has any errors and if it was successfully build and tagged +- Follow steps from [Run, Stop, Remove](#run-stop-remove) +- Follow steps from [Development time cleanup](#Development-time-cleanup) + + [^TOC^](#Table-of-contents) + +#### Composer + +If you want to run any PHP composer command then you have two options: + +exec command: + +```sh +docker-compose exec php-composer-v2 composer install +``` + +or attach to the container: + +```sh +docker attach carrental-php-composer-v2-1 +``` + +then you will be logged into the `carrental-php-composer-v2-1` shell (`/bin/bash`) and inside the project directory so you will be able to type: + +```sh +composer install +``` + +Note that example of attaching above used `docker` command instead of `docker-compose` therefore a full name of the container was used (`carrental-php-composer-v2-1`) instead of the service name (`php-composer-v2`) + +To exit from the attached container just run: + +```sh +exit +``` + +that will exit the container but since its restart policy is: + +```yaml + php-composer-v2: + restart: unless-stopped +``` + +then it will be restarted, allowing you to attach to it again. + + [^TOC^](#Table-of-contents) + +#### Monitoring + +For monitoring is good to open a new terminal window and run: + +```sh +watch -t -n 2 "docker ps -a" +``` + +then open another window and type: + +```sh +docker stats -a +``` + +`docker stats` has issue that after some time there are lot of "dead" entries that are meaningless. In order to get rid of them you need to terminate that command with Ctrl+c and run it again. + +To see logs (in case you run `docker-compose` `up` with the `-d` or `--detach` flag): + +```sh +docker-compose logs +``` + +To list images releated to this project run: + +```sh +docker image ls 'carrental_*' +``` + + [^TOC^](#Table-of-contents) + +#### Reinstallation + +At any point of time you may run the `./setup.sh` script again as described in [Quick setup (without Docker cleanup)](#quick-setup-without-docker-cleanup). + +You may do that for example to change the project's files and directories ownership to a different user. If you want to do that just switch to the user that should own that project and perform the `./setup.sh` again. + + [^TOC^](#Table-of-contents) + +## Uninstallation + +Stop and remove all of the containers: + +```sh +docker-compose down --remove-orphans +``` + + + +list all of the containers' images: + +```sh +docker image ls 'carrental_*' +``` + +If listed images are the one you expect you may remove them by running: + +```sh +docker image rm -f $(docker image ls -q 'carrental_*') +``` + + + +if you see: + +> Error response from daemon: conflict: unable to delete 8a9b37ed9ecc (cannot be forced) - image has dependent child images + + + +remove dandling images: + +```sh +docker rmi $(docker images --filter "dangling=true" -q --no-trunc) +``` + +and repeat command: + +```sh +docker image rm -f $(docker image ls -q 'carrental_*') +``` + +again. + +If you still can't remove an image, try by using its name:tag for eg.: + +```sh +docker image rm -f carrental_php:8.1.3-fpm-buster +``` + + + +Remove the project directory with its files (you should not need `sudo` for this) + + [^TOC^](#Table-of-contents) + +## Elasticsearch + +**References:** + +[Elasticsearch 7.17 Guide](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/index.html) + +[Elasticsearch 7.17 Guide - Docker](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/docker.html) + + + +#### Reinitialize + +At any time when you messed up the example cars index you may re-create it and populate with a single command (your shell needs to be in the [./docker/service](./docker/service) dir): + +```sh +docker-compose exec php-composer-v2 /bin/sh -c "/app/default/.elasticsearch-http-requests/init.sh" +``` + +It deletes index, creates it, and populates with the example data. + +terminal output: + +``` +... + +Performing cURL request: +curl -sSL -X GET 'http://elasticsearch-v7:9200/cars/_count' +{"count":39,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}} +Success. +Done! +``` + +or you may do it step by step: + +- [Delete index](#Delete-index) +- [Create index](#Create-index) +- [Populate index with data](#Populate-index-with-data) + +An alternative way is to run the `./setup.sh` script again as described in [Quick setup (without Docker cleanup)](#quick-setup-without-docker-cleanup) + +You may use also: [Step by step other methods](#Step-by-step-other-methods) + + [^TOC^](#Table-of-contents) + +#### Elasticsearch code -## How to play with it? All Elasticsearch implementation related code is placed in `src/Elasticsearch` directory. The core ranking logic is built [from specific `Factors` classes](https://github.com/lrynek/painless-car-rental/tree/main/src/Elasticsearch/ValueObject/Factor): @@ -68,5 +1550,17 @@ final class RecommendedSorter implements FactorSorterInterface 💡 In order to get rid of customly ranked results on the listing you can switch to `DefaultSorter` that sorts all results ascending by their `id`. + [^TOC^](#Table-of-contents) + +## Credits + +[Ryszard](https://stackoverflow.com/users/1174405?tab=profile) + +- Application conversion to docker-compose +- Setup and initialization scripts +- Docker and docker-compose related manuals + + [^TOC^](#Table-of-contents) + ## Copyrights Apart from [the project's LICENSE](https://github.com/lrynek/painless-car-rental/blob/main/LICENSE), [all car photo samples](https://github.com/lrynek/painless-car-rental/tree/main/public/images/cars) used in the project are taken from Google search results and all copyrights applies to their respective authors and shouldn't be used further than private/educational use without their explicit consent. diff --git a/.elasticsearch-http-requests/Check.http b/app/default/.elasticsearch-http-requests/Check.http similarity index 100% rename from .elasticsearch-http-requests/Check.http rename to app/default/.elasticsearch-http-requests/Check.http diff --git a/.elasticsearch-http-requests/Document/Create_or_Update.http b/app/default/.elasticsearch-http-requests/Document/Create_or_Update.http similarity index 100% rename from .elasticsearch-http-requests/Document/Create_or_Update.http rename to app/default/.elasticsearch-http-requests/Document/Create_or_Update.http diff --git a/.elasticsearch-http-requests/Index/Bulk.http b/app/default/.elasticsearch-http-requests/Index/Bulk.http similarity index 100% rename from .elasticsearch-http-requests/Index/Bulk.http rename to app/default/.elasticsearch-http-requests/Index/Bulk.http diff --git a/.elasticsearch-http-requests/Index/Count.http b/app/default/.elasticsearch-http-requests/Index/Count.http similarity index 100% rename from .elasticsearch-http-requests/Index/Count.http rename to app/default/.elasticsearch-http-requests/Index/Count.http diff --git a/.elasticsearch-http-requests/Index/Create.http b/app/default/.elasticsearch-http-requests/Index/Create.http similarity index 100% rename from .elasticsearch-http-requests/Index/Create.http rename to app/default/.elasticsearch-http-requests/Index/Create.http diff --git a/.elasticsearch-http-requests/Index/Delete.http b/app/default/.elasticsearch-http-requests/Index/Delete.http similarity index 100% rename from .elasticsearch-http-requests/Index/Delete.http rename to app/default/.elasticsearch-http-requests/Index/Delete.http diff --git a/.elasticsearch-http-requests/Index/Mappings.http b/app/default/.elasticsearch-http-requests/Index/Mappings.http similarity index 100% rename from .elasticsearch-http-requests/Index/Mappings.http rename to app/default/.elasticsearch-http-requests/Index/Mappings.http diff --git a/.elasticsearch-http-requests/Search/All_structure.http b/app/default/.elasticsearch-http-requests/Search/All_structure.http similarity index 100% rename from .elasticsearch-http-requests/Search/All_structure.http rename to app/default/.elasticsearch-http-requests/Search/All_structure.http diff --git a/.elasticsearch-http-requests/Search/Functions.http b/app/default/.elasticsearch-http-requests/Search/Functions.http similarity index 100% rename from .elasticsearch-http-requests/Search/Functions.http rename to app/default/.elasticsearch-http-requests/Search/Functions.http diff --git a/.elasticsearch-http-requests/Search/Functions_colors.http b/app/default/.elasticsearch-http-requests/Search/Functions_colors.http similarity index 100% rename from .elasticsearch-http-requests/Search/Functions_colors.http rename to app/default/.elasticsearch-http-requests/Search/Functions_colors.http diff --git a/.elasticsearch-http-requests/Search/Phrase.http b/app/default/.elasticsearch-http-requests/Search/Phrase.http similarity index 100% rename from .elasticsearch-http-requests/Search/Phrase.http rename to app/default/.elasticsearch-http-requests/Search/Phrase.http diff --git a/app/default/.elasticsearch-http-requests/init.sh b/app/default/.elasticsearch-http-requests/init.sh new file mode 100755 index 0000000..316e209 --- /dev/null +++ b/app/default/.elasticsearch-http-requests/init.sh @@ -0,0 +1,12 @@ +#!/bin/sh + +set -eu + +SCRIPTPATH="$( cd "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )"; + +"$SCRIPTPATH/request.sh" "$SCRIPTPATH/Index/Delete.http" || true +"$SCRIPTPATH/request.sh" "$SCRIPTPATH/Index/Create.http" && \ +"$SCRIPTPATH/request.sh" "$SCRIPTPATH/Index/Bulk.http" && sleep 1 && \ +"$SCRIPTPATH/request.sh" "$SCRIPTPATH/Index/Count.http" + +printf "Done!\n" diff --git a/app/default/.elasticsearch-http-requests/request.sh b/app/default/.elasticsearch-http-requests/request.sh new file mode 100755 index 0000000..55730db --- /dev/null +++ b/app/default/.elasticsearch-http-requests/request.sh @@ -0,0 +1,84 @@ +#!/bin/sh + +# Example usage: +# ./request.sh ./Index/Create.http +# ./request.sh /app/default/.elasticsearch-http-requests/Index/Delete.http + +set -eu + +http_request_path="${1:-}" + +[ -n "$http_request_path" ] || \ + { printf "Usage: $1 \n" && exit 1; } + +[ -n "${DOCKER_ELASTICSEARCH_HOST:-}" ] || \ + { printf "Set ENV varialbe DOCKER_ELASTICSEARCH_HOST first\n" && exit 1 ;} + +RED='\033[31;5;7m' +GREEN='\033[1;0;42m' +NC='\033[0m' # No Color + +SCRIPTPATH="$( cd "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )"; + +# if request path is not absolute make it relative to the script +[ ! "$http_request_path" = "${http_request_path#/}" ] || \ + http_request_path="$SCRIPTPATH/$http_request_path" + +[ -f $http_request_path ] || \ + { printf "Path does not point to a file, path: $1\n" && exit 1; } + +[ -r "$http_request_path" ] || \ + { printf "File is not readable, check permissions, path: $1\n" && exit 1; } + +# read content of a file +content="$(cat "$http_request_path" || { 1>&2 printf "Could not read the file: $http_request_path\n" && exit 1; })" + +# dissasemble request to its part +request="$(printf "$content" | head -n1)" +method="$(printf "$request" | grep -oE '^\w+')" +url=$(printf "$request" | sed 's/^\w* //') +header="$(printf "$content\n" | awk '/http/,/^$/' | tail -n+2 | sed '$d')" +body="$(printf "$content" | tail -n+"$(printf "$content\n" | awk '/http/,/^$/' | wc -l)" | tail -n+2)" + +# convert localhost to elasticsearch service host +find='http://localhost:9200' +replace="$DOCKER_ELASTICSEARCH_HOST" +url="$(printf "$url\n" | awk -v r="$replace" -v f="$find" '{ gsub(f,r); print $0 }')" + +# create header options for curl +curl_header='' +newline="$(printf "\nx")" && newline="${newline%x}" +IFS="$newline" +for head in $header; do + curl_header="${curl_header} -H '$head'" +done + +option_data='' +[ -z "$body" ] || option_data=" --data-raw \\${newline}'$body${newline}'" + +# escape body's single quote ' +body="$(printf "$body" | sed "s/'/\\\'/g")" + +from_newline='' +[ -z "$option_data" ] || from_newline="\\${newline}" + +#Bugfix for: elasticsearch 7 deprec. use of _doc: '{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true."}],"type":"illegal_argument_exception","reason":"The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true."},"status":400}' +[ ! "$method" = 'PUT' ] || url="${url}?include_type_name=true" + +command_request="curl -sSL -X ${method:-GET}${curl_header}${option_data} ${from_newline}'$url'" +printf "Performing cURL request:\n${command_request}\n" + +# example success responses +# expected_pattern='\{"count":0,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}}' +# expected_pattern='^\{"acknowledged":true' # Delete, + +# check for error response +expected_pattern='^\s*\{\s*"error"' # Delete, +response="$(printf "$command_request\n" | sh || ec=$? )" + +# validate response +printf "$response" | grep -vE "$expected_pattern" > /dev/null || \ + { printf "${RED}FAIL${NC}\nThere was an error with cURL exit code: ${ec:-0}, and response: $response\n" && exit ${ec:-1}; } + +printf "$response\n" +printf "${GREEN}Success.${NC}\n" diff --git a/src/Controller/.gitignore b/app/default/.env similarity index 100% rename from src/Controller/.gitignore rename to app/default/.env diff --git a/.gitignore b/app/default/.gitignore similarity index 100% rename from .gitignore rename to app/default/.gitignore diff --git a/app/default/LICENSE b/app/default/LICENSE new file mode 100644 index 0000000..1c9cbe2 --- /dev/null +++ b/app/default/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021 Łukasz Rynek + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/bin/console b/app/default/bin/console old mode 100755 new mode 100644 similarity index 100% rename from bin/console rename to app/default/bin/console diff --git a/composer.json b/app/default/composer.json similarity index 100% rename from composer.json rename to app/default/composer.json diff --git a/composer.lock b/app/default/composer.lock similarity index 100% rename from composer.lock rename to app/default/composer.lock diff --git a/config/bundles.php b/app/default/config/bundles.php similarity index 100% rename from config/bundles.php rename to app/default/config/bundles.php diff --git a/config/packages/cache.yaml b/app/default/config/packages/cache.yaml similarity index 100% rename from config/packages/cache.yaml rename to app/default/config/packages/cache.yaml diff --git a/config/packages/framework.yaml b/app/default/config/packages/framework.yaml similarity index 100% rename from config/packages/framework.yaml rename to app/default/config/packages/framework.yaml diff --git a/config/packages/routing.yaml b/app/default/config/packages/routing.yaml similarity index 100% rename from config/packages/routing.yaml rename to app/default/config/packages/routing.yaml diff --git a/config/packages/sensio_framework_extra.yaml b/app/default/config/packages/sensio_framework_extra.yaml similarity index 100% rename from config/packages/sensio_framework_extra.yaml rename to app/default/config/packages/sensio_framework_extra.yaml diff --git a/config/packages/twig.yaml b/app/default/config/packages/twig.yaml similarity index 100% rename from config/packages/twig.yaml rename to app/default/config/packages/twig.yaml diff --git a/config/preload.php b/app/default/config/preload.php similarity index 100% rename from config/preload.php rename to app/default/config/preload.php diff --git a/config/routes.yaml b/app/default/config/routes.yaml similarity index 100% rename from config/routes.yaml rename to app/default/config/routes.yaml diff --git a/config/routes/annotations.yaml b/app/default/config/routes/annotations.yaml similarity index 100% rename from config/routes/annotations.yaml rename to app/default/config/routes/annotations.yaml diff --git a/config/routes/framework.yaml b/app/default/config/routes/framework.yaml similarity index 100% rename from config/routes/framework.yaml rename to app/default/config/routes/framework.yaml diff --git a/config/services.yaml b/app/default/config/services.yaml similarity index 88% rename from config/services.yaml rename to app/default/config/services.yaml index f222cf4..242c2cd 100644 --- a/config/services.yaml +++ b/app/default/config/services.yaml @@ -4,6 +4,7 @@ # Put parameters here that don't need to change on each machine where the app is deployed # https://symfony.com/doc/current/best_practices.html#use-parameters-for-application-configuration parameters: + app.elasticsearch.host: '%env(DOCKER_ELASTICSEARCH_HOST)%' services: # default configuration for services in *this* file @@ -35,3 +36,7 @@ services: resource: '../src/Twig/Extension/*' tags: - { name: twig.extension } + + App\Elasticsearch\Service\ApiClient: + arguments: + $elasticsearchHost: '%app.elasticsearch.host%' diff --git a/insomnia.json b/app/default/insomnia.json similarity index 100% rename from insomnia.json rename to app/default/insomnia.json diff --git a/public/bootstrap.bundle.min.js b/app/default/public/bootstrap.bundle.min.js similarity index 100% rename from public/bootstrap.bundle.min.js rename to app/default/public/bootstrap.bundle.min.js diff --git a/public/bootstrap.min.css b/app/default/public/bootstrap.min.css similarity index 100% rename from public/bootstrap.min.css rename to app/default/public/bootstrap.min.css diff --git a/public/favicon.ico b/app/default/public/favicon.ico similarity index 100% rename from public/favicon.ico rename to app/default/public/favicon.ico diff --git a/public/favicon.png b/app/default/public/favicon.png similarity index 100% rename from public/favicon.png rename to app/default/public/favicon.png diff --git a/public/images/cars/aston-martin-db7-gt.jpg b/app/default/public/images/cars/aston-martin-db7-gt.jpg similarity index 100% rename from public/images/cars/aston-martin-db7-gt.jpg rename to app/default/public/images/cars/aston-martin-db7-gt.jpg diff --git a/public/images/cars/aston-martin-db7-vantage.jpg b/app/default/public/images/cars/aston-martin-db7-vantage.jpg similarity index 100% rename from public/images/cars/aston-martin-db7-vantage.jpg rename to app/default/public/images/cars/aston-martin-db7-vantage.jpg diff --git a/public/images/cars/aston-martin-db7-zagato.jpg b/app/default/public/images/cars/aston-martin-db7-zagato.jpg similarity index 100% rename from public/images/cars/aston-martin-db7-zagato.jpg rename to app/default/public/images/cars/aston-martin-db7-zagato.jpg diff --git a/public/images/cars/bmw-i8-roadster-2019.jpg b/app/default/public/images/cars/bmw-i8-roadster-2019.jpg similarity index 100% rename from public/images/cars/bmw-i8-roadster-2019.jpg rename to app/default/public/images/cars/bmw-i8-roadster-2019.jpg diff --git a/public/images/cars/bmw-i8-roadster.jpg b/app/default/public/images/cars/bmw-i8-roadster.jpg similarity index 100% rename from public/images/cars/bmw-i8-roadster.jpg rename to app/default/public/images/cars/bmw-i8-roadster.jpg diff --git a/public/images/cars/dodge-challenger.jpg b/app/default/public/images/cars/dodge-challenger.jpg similarity index 100% rename from public/images/cars/dodge-challenger.jpg rename to app/default/public/images/cars/dodge-challenger.jpg diff --git a/public/images/cars/dodge-viper-gts.jpg b/app/default/public/images/cars/dodge-viper-gts.jpg similarity index 100% rename from public/images/cars/dodge-viper-gts.jpg rename to app/default/public/images/cars/dodge-viper-gts.jpg diff --git a/public/images/cars/dodge-viper-srt10-roadster.jpg b/app/default/public/images/cars/dodge-viper-srt10-roadster.jpg similarity index 100% rename from public/images/cars/dodge-viper-srt10-roadster.jpg rename to app/default/public/images/cars/dodge-viper-srt10-roadster.jpg diff --git a/public/images/cars/ferrari-250-gt-california.jpg b/app/default/public/images/cars/ferrari-250-gt-california.jpg similarity index 100% rename from public/images/cars/ferrari-250-gt-california.jpg rename to app/default/public/images/cars/ferrari-250-gt-california.jpg diff --git a/public/images/cars/ferrari-296-gtb.jpg b/app/default/public/images/cars/ferrari-296-gtb.jpg similarity index 100% rename from public/images/cars/ferrari-296-gtb.jpg rename to app/default/public/images/cars/ferrari-296-gtb.jpg diff --git a/public/images/cars/ferrari-550-maranello.jpg b/app/default/public/images/cars/ferrari-550-maranello.jpg similarity index 100% rename from public/images/cars/ferrari-550-maranello.jpg rename to app/default/public/images/cars/ferrari-550-maranello.jpg diff --git a/public/images/cars/ferrari-f8-spider.jpg b/app/default/public/images/cars/ferrari-f8-spider.jpg similarity index 100% rename from public/images/cars/ferrari-f8-spider.jpg rename to app/default/public/images/cars/ferrari-f8-spider.jpg diff --git a/public/images/cars/ferrari-roma.jpg b/app/default/public/images/cars/ferrari-roma.jpg similarity index 100% rename from public/images/cars/ferrari-roma.jpg rename to app/default/public/images/cars/ferrari-roma.jpg diff --git a/public/images/cars/ferrari-sf90.jpg b/app/default/public/images/cars/ferrari-sf90.jpg similarity index 100% rename from public/images/cars/ferrari-sf90.jpg rename to app/default/public/images/cars/ferrari-sf90.jpg diff --git a/public/images/cars/ferrari-testarossa.jpg b/app/default/public/images/cars/ferrari-testarossa.jpg similarity index 100% rename from public/images/cars/ferrari-testarossa.jpg rename to app/default/public/images/cars/ferrari-testarossa.jpg diff --git a/public/images/cars/ford-gt.jpg b/app/default/public/images/cars/ford-gt.jpg similarity index 100% rename from public/images/cars/ford-gt.jpg rename to app/default/public/images/cars/ford-gt.jpg diff --git a/public/images/cars/ford-mustang-boss-302.jpg b/app/default/public/images/cars/ford-mustang-boss-302.jpg similarity index 100% rename from public/images/cars/ford-mustang-boss-302.jpg rename to app/default/public/images/cars/ford-mustang-boss-302.jpg diff --git a/public/images/cars/ford-mustang-rtr.jpg b/app/default/public/images/cars/ford-mustang-rtr.jpg similarity index 100% rename from public/images/cars/ford-mustang-rtr.jpg rename to app/default/public/images/cars/ford-mustang-rtr.jpg diff --git a/public/images/cars/lamborghini-aventador.jpg b/app/default/public/images/cars/lamborghini-aventador.jpg similarity index 100% rename from public/images/cars/lamborghini-aventador.jpg rename to app/default/public/images/cars/lamborghini-aventador.jpg diff --git a/public/images/cars/lamborghini-countach.jpg b/app/default/public/images/cars/lamborghini-countach.jpg similarity index 100% rename from public/images/cars/lamborghini-countach.jpg rename to app/default/public/images/cars/lamborghini-countach.jpg diff --git a/public/images/cars/lamborghini-diablo.jpg b/app/default/public/images/cars/lamborghini-diablo.jpg similarity index 100% rename from public/images/cars/lamborghini-diablo.jpg rename to app/default/public/images/cars/lamborghini-diablo.jpg diff --git a/public/images/cars/lamborghini-huracan.jpg b/app/default/public/images/cars/lamborghini-huracan.jpg similarity index 100% rename from public/images/cars/lamborghini-huracan.jpg rename to app/default/public/images/cars/lamborghini-huracan.jpg diff --git a/public/images/cars/lamborghini-murcielago-lp670.jpg b/app/default/public/images/cars/lamborghini-murcielago-lp670.jpg similarity index 100% rename from public/images/cars/lamborghini-murcielago-lp670.jpg rename to app/default/public/images/cars/lamborghini-murcielago-lp670.jpg diff --git a/public/images/cars/lamborghini-sc20.jpg b/app/default/public/images/cars/lamborghini-sc20.jpg similarity index 100% rename from public/images/cars/lamborghini-sc20.jpg rename to app/default/public/images/cars/lamborghini-sc20.jpg diff --git a/public/images/cars/mazda-mx5-miata.jpg b/app/default/public/images/cars/mazda-mx5-miata.jpg similarity index 100% rename from public/images/cars/mazda-mx5-miata.jpg rename to app/default/public/images/cars/mazda-mx5-miata.jpg diff --git a/public/images/cars/mazda-mx5-z-sport.jpg b/app/default/public/images/cars/mazda-mx5-z-sport.jpg similarity index 100% rename from public/images/cars/mazda-mx5-z-sport.jpg rename to app/default/public/images/cars/mazda-mx5-z-sport.jpg diff --git a/public/images/cars/mazda-rx-vision-gt3-concept.jpg b/app/default/public/images/cars/mazda-rx-vision-gt3-concept.jpg similarity index 100% rename from public/images/cars/mazda-rx-vision-gt3-concept.jpg rename to app/default/public/images/cars/mazda-rx-vision-gt3-concept.jpg diff --git a/public/images/cars/mclaren-570s.jpg b/app/default/public/images/cars/mclaren-570s.jpg similarity index 100% rename from public/images/cars/mclaren-570s.jpg rename to app/default/public/images/cars/mclaren-570s.jpg diff --git a/public/images/cars/mclaren-720s.jpg b/app/default/public/images/cars/mclaren-720s.jpg similarity index 100% rename from public/images/cars/mclaren-720s.jpg rename to app/default/public/images/cars/mclaren-720s.jpg diff --git a/public/images/cars/mclaren-f1.jpg b/app/default/public/images/cars/mclaren-f1.jpg similarity index 100% rename from public/images/cars/mclaren-f1.jpg rename to app/default/public/images/cars/mclaren-f1.jpg diff --git a/public/images/cars/mercedes-benz-amg-gt-coupe.jpg b/app/default/public/images/cars/mercedes-benz-amg-gt-coupe.jpg similarity index 100% rename from public/images/cars/mercedes-benz-amg-gt-coupe.jpg rename to app/default/public/images/cars/mercedes-benz-amg-gt-coupe.jpg diff --git a/public/images/cars/mercedes-benz-gt.jpg b/app/default/public/images/cars/mercedes-benz-gt.jpg similarity index 100% rename from public/images/cars/mercedes-benz-gt.jpg rename to app/default/public/images/cars/mercedes-benz-gt.jpg diff --git a/public/images/cars/mercedes-benz-sls-amg-coupe.jpg b/app/default/public/images/cars/mercedes-benz-sls-amg-coupe.jpg similarity index 100% rename from public/images/cars/mercedes-benz-sls-amg-coupe.jpg rename to app/default/public/images/cars/mercedes-benz-sls-amg-coupe.jpg diff --git a/public/images/cars/mercedes-mclaren-slr-722-stirling-moss.jpg b/app/default/public/images/cars/mercedes-mclaren-slr-722-stirling-moss.jpg similarity index 100% rename from public/images/cars/mercedes-mclaren-slr-722-stirling-moss.jpg rename to app/default/public/images/cars/mercedes-mclaren-slr-722-stirling-moss.jpg diff --git a/public/images/cars/mitsubishi-lancer-evo-x.jpg b/app/default/public/images/cars/mitsubishi-lancer-evo-x.jpg similarity index 100% rename from public/images/cars/mitsubishi-lancer-evo-x.jpg rename to app/default/public/images/cars/mitsubishi-lancer-evo-x.jpg diff --git a/public/images/cars/porshe-911-gts.jpg b/app/default/public/images/cars/porshe-911-gts.jpg similarity index 100% rename from public/images/cars/porshe-911-gts.jpg rename to app/default/public/images/cars/porshe-911-gts.jpg diff --git a/public/images/cars/porshe-911-swb.jpg b/app/default/public/images/cars/porshe-911-swb.jpg similarity index 100% rename from public/images/cars/porshe-911-swb.jpg rename to app/default/public/images/cars/porshe-911-swb.jpg diff --git a/public/images/cars/porshe-911-turbo-s.jpg b/app/default/public/images/cars/porshe-911-turbo-s.jpg similarity index 100% rename from public/images/cars/porshe-911-turbo-s.jpg rename to app/default/public/images/cars/porshe-911-turbo-s.jpg diff --git a/public/images/cars/porshe-macan.jpg b/app/default/public/images/cars/porshe-macan.jpg similarity index 100% rename from public/images/cars/porshe-macan.jpg rename to app/default/public/images/cars/porshe-macan.jpg diff --git a/public/index.php b/app/default/public/index.php similarity index 100% rename from public/index.php rename to app/default/public/index.php diff --git a/public/jquery.min.js b/app/default/public/jquery.min.js similarity index 100% rename from public/jquery.min.js rename to app/default/public/jquery.min.js diff --git a/public/main.css b/app/default/public/main.css similarity index 100% rename from public/main.css rename to app/default/public/main.css diff --git a/public/select2.min.css b/app/default/public/select2.min.css similarity index 100% rename from public/select2.min.css rename to app/default/public/select2.min.css diff --git a/public/select2.min.js b/app/default/public/select2.min.js similarity index 100% rename from public/select2.min.js rename to app/default/public/select2.min.js diff --git a/app/default/src/Controller/.gitignore b/app/default/src/Controller/.gitignore new file mode 100644 index 0000000..e69de29 diff --git a/src/Controller/DefaultController.php b/app/default/src/Controller/DefaultController.php similarity index 100% rename from src/Controller/DefaultController.php rename to app/default/src/Controller/DefaultController.php diff --git a/src/Controller/ParamConverter/AbstractParamConverter.php b/app/default/src/Controller/ParamConverter/AbstractParamConverter.php similarity index 100% rename from src/Controller/ParamConverter/AbstractParamConverter.php rename to app/default/src/Controller/ParamConverter/AbstractParamConverter.php diff --git a/src/Controller/ParamConverter/CriteriaParamConverter.php b/app/default/src/Controller/ParamConverter/CriteriaParamConverter.php similarity index 100% rename from src/Controller/ParamConverter/CriteriaParamConverter.php rename to app/default/src/Controller/ParamConverter/CriteriaParamConverter.php diff --git a/src/Controller/ParamConverter/PageParamConverter.php b/app/default/src/Controller/ParamConverter/PageParamConverter.php similarity index 100% rename from src/Controller/ParamConverter/PageParamConverter.php rename to app/default/src/Controller/ParamConverter/PageParamConverter.php diff --git a/src/Elasticsearch/Hydrator/CarHydrator.php b/app/default/src/Elasticsearch/Hydrator/CarHydrator.php similarity index 100% rename from src/Elasticsearch/Hydrator/CarHydrator.php rename to app/default/src/Elasticsearch/Hydrator/CarHydrator.php diff --git a/src/Elasticsearch/Hydrator/CarHydratorInterface.php b/app/default/src/Elasticsearch/Hydrator/CarHydratorInterface.php similarity index 100% rename from src/Elasticsearch/Hydrator/CarHydratorInterface.php rename to app/default/src/Elasticsearch/Hydrator/CarHydratorInterface.php diff --git a/src/Elasticsearch/Hydrator/CarsHydrator.php b/app/default/src/Elasticsearch/Hydrator/CarsHydrator.php similarity index 100% rename from src/Elasticsearch/Hydrator/CarsHydrator.php rename to app/default/src/Elasticsearch/Hydrator/CarsHydrator.php diff --git a/src/Elasticsearch/Hydrator/CarsHydratorInterface.php b/app/default/src/Elasticsearch/Hydrator/CarsHydratorInterface.php similarity index 100% rename from src/Elasticsearch/Hydrator/CarsHydratorInterface.php rename to app/default/src/Elasticsearch/Hydrator/CarsHydratorInterface.php diff --git a/src/Elasticsearch/Repository/CarRepository.php b/app/default/src/Elasticsearch/Repository/CarRepository.php similarity index 100% rename from src/Elasticsearch/Repository/CarRepository.php rename to app/default/src/Elasticsearch/Repository/CarRepository.php diff --git a/src/Elasticsearch/Service/ApiClient.php b/app/default/src/Elasticsearch/Service/ApiClient.php similarity index 83% rename from src/Elasticsearch/Service/ApiClient.php rename to app/default/src/Elasticsearch/Service/ApiClient.php index 80c29f6..0215fed 100644 --- a/src/Elasticsearch/Service/ApiClient.php +++ b/app/default/src/Elasticsearch/Service/ApiClient.php @@ -11,12 +11,13 @@ final class ApiClient implements ApiClientInterface { - private const ELASTICSEARCH_HOST = 'http://localhost:9200'; + private $elasticsearchHost; private const ENDPOINT_SEARCH = '_search'; - public function __construct(private ClientInterface $client) + public function __construct(private ClientInterface $client, string $elasticsearchHost) { + $this->elasticsearchHost = $elasticsearchHost; } public function search(Index $index, Query $query): Response @@ -29,7 +30,7 @@ public function search(Index $index, Query $query): Response private function createEndpointUrl(string $index, string $endpoint): string { - return implode(DIRECTORY_SEPARATOR, [self::ELASTICSEARCH_HOST, $index, $endpoint]); + return implode(DIRECTORY_SEPARATOR, [$this->elasticsearchHost, $index, $endpoint]); } private function convertResponseToArray(ResponseInterface $response): array diff --git a/src/Elasticsearch/Service/ApiClientInterface.php b/app/default/src/Elasticsearch/Service/ApiClientInterface.php similarity index 100% rename from src/Elasticsearch/Service/ApiClientInterface.php rename to app/default/src/Elasticsearch/Service/ApiClientInterface.php diff --git a/src/Elasticsearch/ValueObject/Criteria/ColorsFilter.php b/app/default/src/Elasticsearch/ValueObject/Criteria/ColorsFilter.php similarity index 100% rename from src/Elasticsearch/ValueObject/Criteria/ColorsFilter.php rename to app/default/src/Elasticsearch/ValueObject/Criteria/ColorsFilter.php diff --git a/src/Elasticsearch/ValueObject/Criteria/Criteria.php b/app/default/src/Elasticsearch/ValueObject/Criteria/Criteria.php similarity index 100% rename from src/Elasticsearch/ValueObject/Criteria/Criteria.php rename to app/default/src/Elasticsearch/ValueObject/Criteria/Criteria.php diff --git a/src/Elasticsearch/ValueObject/Criteria/Criterion.php b/app/default/src/Elasticsearch/ValueObject/Criteria/Criterion.php similarity index 100% rename from src/Elasticsearch/ValueObject/Criteria/Criterion.php rename to app/default/src/Elasticsearch/ValueObject/Criteria/Criterion.php diff --git a/src/Elasticsearch/ValueObject/Criteria/PhraseFilter.php b/app/default/src/Elasticsearch/ValueObject/Criteria/PhraseFilter.php similarity index 100% rename from src/Elasticsearch/ValueObject/Criteria/PhraseFilter.php rename to app/default/src/Elasticsearch/ValueObject/Criteria/PhraseFilter.php diff --git a/src/Elasticsearch/ValueObject/Criteria/ServicesFilter.php b/app/default/src/Elasticsearch/ValueObject/Criteria/ServicesFilter.php similarity index 100% rename from src/Elasticsearch/ValueObject/Criteria/ServicesFilter.php rename to app/default/src/Elasticsearch/ValueObject/Criteria/ServicesFilter.php diff --git a/src/Elasticsearch/ValueObject/Factor/ColorRelevanceFactor.php b/app/default/src/Elasticsearch/ValueObject/Factor/ColorRelevanceFactor.php similarity index 100% rename from src/Elasticsearch/ValueObject/Factor/ColorRelevanceFactor.php rename to app/default/src/Elasticsearch/ValueObject/Factor/ColorRelevanceFactor.php diff --git a/src/Elasticsearch/ValueObject/Factor/DodgePromoFactor.php b/app/default/src/Elasticsearch/ValueObject/Factor/DodgePromoFactor.php similarity index 100% rename from src/Elasticsearch/ValueObject/Factor/DodgePromoFactor.php rename to app/default/src/Elasticsearch/ValueObject/Factor/DodgePromoFactor.php diff --git a/src/Elasticsearch/ValueObject/Factor/FactorInterface.php b/app/default/src/Elasticsearch/ValueObject/Factor/FactorInterface.php similarity index 100% rename from src/Elasticsearch/ValueObject/Factor/FactorInterface.php rename to app/default/src/Elasticsearch/ValueObject/Factor/FactorInterface.php diff --git a/src/Elasticsearch/ValueObject/Factor/Factors.php b/app/default/src/Elasticsearch/ValueObject/Factor/Factors.php similarity index 100% rename from src/Elasticsearch/ValueObject/Factor/Factors.php rename to app/default/src/Elasticsearch/ValueObject/Factor/Factors.php diff --git a/src/Elasticsearch/ValueObject/Factor/RawScoreFactor.php b/app/default/src/Elasticsearch/ValueObject/Factor/RawScoreFactor.php similarity index 100% rename from src/Elasticsearch/ValueObject/Factor/RawScoreFactor.php rename to app/default/src/Elasticsearch/ValueObject/Factor/RawScoreFactor.php diff --git a/src/Elasticsearch/ValueObject/Factor/Weight.php b/app/default/src/Elasticsearch/ValueObject/Factor/Weight.php similarity index 100% rename from src/Elasticsearch/ValueObject/Factor/Weight.php rename to app/default/src/Elasticsearch/ValueObject/Factor/Weight.php diff --git a/src/Elasticsearch/ValueObject/Factor/WeightFactor.php b/app/default/src/Elasticsearch/ValueObject/Factor/WeightFactor.php similarity index 100% rename from src/Elasticsearch/ValueObject/Factor/WeightFactor.php rename to app/default/src/Elasticsearch/ValueObject/Factor/WeightFactor.php diff --git a/src/Elasticsearch/ValueObject/Query.php b/app/default/src/Elasticsearch/ValueObject/Query.php similarity index 100% rename from src/Elasticsearch/ValueObject/Query.php rename to app/default/src/Elasticsearch/ValueObject/Query.php diff --git a/src/Elasticsearch/ValueObject/Response.php b/app/default/src/Elasticsearch/ValueObject/Response.php similarity index 100% rename from src/Elasticsearch/ValueObject/Response.php rename to app/default/src/Elasticsearch/ValueObject/Response.php diff --git a/src/Elasticsearch/ValueObject/Result.php b/app/default/src/Elasticsearch/ValueObject/Result.php similarity index 100% rename from src/Elasticsearch/ValueObject/Result.php rename to app/default/src/Elasticsearch/ValueObject/Result.php diff --git a/src/Elasticsearch/ValueObject/Results.php b/app/default/src/Elasticsearch/ValueObject/Results.php similarity index 100% rename from src/Elasticsearch/ValueObject/Results.php rename to app/default/src/Elasticsearch/ValueObject/Results.php diff --git a/src/Elasticsearch/ValueObject/Sorter/DefaultSorter.php b/app/default/src/Elasticsearch/ValueObject/Sorter/DefaultSorter.php similarity index 100% rename from src/Elasticsearch/ValueObject/Sorter/DefaultSorter.php rename to app/default/src/Elasticsearch/ValueObject/Sorter/DefaultSorter.php diff --git a/src/Elasticsearch/ValueObject/Sorter/FactorSorterInterface.php b/app/default/src/Elasticsearch/ValueObject/Sorter/FactorSorterInterface.php similarity index 100% rename from src/Elasticsearch/ValueObject/Sorter/FactorSorterInterface.php rename to app/default/src/Elasticsearch/ValueObject/Sorter/FactorSorterInterface.php diff --git a/src/Elasticsearch/ValueObject/Sorter/RecommendedSorter.php b/app/default/src/Elasticsearch/ValueObject/Sorter/RecommendedSorter.php similarity index 100% rename from src/Elasticsearch/ValueObject/Sorter/RecommendedSorter.php rename to app/default/src/Elasticsearch/ValueObject/Sorter/RecommendedSorter.php diff --git a/src/Elasticsearch/ValueObject/Sorter/SorterInterface.php b/app/default/src/Elasticsearch/ValueObject/Sorter/SorterInterface.php similarity index 100% rename from src/Elasticsearch/ValueObject/Sorter/SorterInterface.php rename to app/default/src/Elasticsearch/ValueObject/Sorter/SorterInterface.php diff --git a/src/Enum/AbstractEnum.php b/app/default/src/Enum/AbstractEnum.php similarity index 100% rename from src/Enum/AbstractEnum.php rename to app/default/src/Enum/AbstractEnum.php diff --git a/src/Enum/Color.php b/app/default/src/Enum/Color.php similarity index 100% rename from src/Enum/Color.php rename to app/default/src/Enum/Color.php diff --git a/src/Enum/Index.php b/app/default/src/Enum/Index.php similarity index 100% rename from src/Enum/Index.php rename to app/default/src/Enum/Index.php diff --git a/src/EventSubscriber/PaginationEventSubscriber.php b/app/default/src/EventSubscriber/PaginationEventSubscriber.php similarity index 100% rename from src/EventSubscriber/PaginationEventSubscriber.php rename to app/default/src/EventSubscriber/PaginationEventSubscriber.php diff --git a/src/Kernel.php b/app/default/src/Kernel.php similarity index 100% rename from src/Kernel.php rename to app/default/src/Kernel.php diff --git a/src/Repository/AdditionalServiceRepositoryInterface.php b/app/default/src/Repository/AdditionalServiceRepositoryInterface.php similarity index 100% rename from src/Repository/AdditionalServiceRepositoryInterface.php rename to app/default/src/Repository/AdditionalServiceRepositoryInterface.php diff --git a/src/Repository/CarRepositoryInterface.php b/app/default/src/Repository/CarRepositoryInterface.php similarity index 100% rename from src/Repository/CarRepositoryInterface.php rename to app/default/src/Repository/CarRepositoryInterface.php diff --git a/src/Repository/InMemoryAdditionalServiceRepository.php b/app/default/src/Repository/InMemoryAdditionalServiceRepository.php similarity index 100% rename from src/Repository/InMemoryAdditionalServiceRepository.php rename to app/default/src/Repository/InMemoryAdditionalServiceRepository.php diff --git a/src/Twig/Extension/AppExtension.php b/app/default/src/Twig/Extension/AppExtension.php similarity index 100% rename from src/Twig/Extension/AppExtension.php rename to app/default/src/Twig/Extension/AppExtension.php diff --git a/src/ValueObject/AdditionalService.php b/app/default/src/ValueObject/AdditionalService.php similarity index 100% rename from src/ValueObject/AdditionalService.php rename to app/default/src/ValueObject/AdditionalService.php diff --git a/src/ValueObject/AdditionalServices.php b/app/default/src/ValueObject/AdditionalServices.php similarity index 100% rename from src/ValueObject/AdditionalServices.php rename to app/default/src/ValueObject/AdditionalServices.php diff --git a/src/ValueObject/Car.php b/app/default/src/ValueObject/Car.php similarity index 100% rename from src/ValueObject/Car.php rename to app/default/src/ValueObject/Car.php diff --git a/src/ValueObject/Cars.php b/app/default/src/ValueObject/Cars.php similarity index 100% rename from src/ValueObject/Cars.php rename to app/default/src/ValueObject/Cars.php diff --git a/src/ValueObject/Colors.php b/app/default/src/ValueObject/Colors.php similarity index 100% rename from src/ValueObject/Colors.php rename to app/default/src/ValueObject/Colors.php diff --git a/src/ValueObject/CriteriaInterface.php b/app/default/src/ValueObject/CriteriaInterface.php similarity index 100% rename from src/ValueObject/CriteriaInterface.php rename to app/default/src/ValueObject/CriteriaInterface.php diff --git a/src/ValueObject/CriterionInterface.php b/app/default/src/ValueObject/CriterionInterface.php similarity index 100% rename from src/ValueObject/CriterionInterface.php rename to app/default/src/ValueObject/CriterionInterface.php diff --git a/src/ValueObject/Id.php b/app/default/src/ValueObject/Id.php similarity index 100% rename from src/ValueObject/Id.php rename to app/default/src/ValueObject/Id.php diff --git a/src/ValueObject/Model.php b/app/default/src/ValueObject/Model.php similarity index 100% rename from src/ValueObject/Model.php rename to app/default/src/ValueObject/Model.php diff --git a/src/ValueObject/Name.php b/app/default/src/ValueObject/Name.php similarity index 100% rename from src/ValueObject/Name.php rename to app/default/src/ValueObject/Name.php diff --git a/src/ValueObject/Page.php b/app/default/src/ValueObject/Page.php similarity index 100% rename from src/ValueObject/Page.php rename to app/default/src/ValueObject/Page.php diff --git a/src/ValueObject/PagesTotal.php b/app/default/src/ValueObject/PagesTotal.php similarity index 100% rename from src/ValueObject/PagesTotal.php rename to app/default/src/ValueObject/PagesTotal.php diff --git a/src/ValueObject/PagesTotalAwareInterface.php b/app/default/src/ValueObject/PagesTotalAwareInterface.php similarity index 100% rename from src/ValueObject/PagesTotalAwareInterface.php rename to app/default/src/ValueObject/PagesTotalAwareInterface.php diff --git a/src/ValueObject/Pagination.php b/app/default/src/ValueObject/Pagination.php similarity index 100% rename from src/ValueObject/Pagination.php rename to app/default/src/ValueObject/Pagination.php diff --git a/src/ValueObject/Phrase.php b/app/default/src/ValueObject/Phrase.php similarity index 100% rename from src/ValueObject/Phrase.php rename to app/default/src/ValueObject/Phrase.php diff --git a/src/ValueObject/Picture.php b/app/default/src/ValueObject/Picture.php similarity index 100% rename from src/ValueObject/Picture.php rename to app/default/src/ValueObject/Picture.php diff --git a/src/ValueObject/Producer.php b/app/default/src/ValueObject/Producer.php similarity index 100% rename from src/ValueObject/Producer.php rename to app/default/src/ValueObject/Producer.php diff --git a/src/ValueObject/ProductionYear.php b/app/default/src/ValueObject/ProductionYear.php similarity index 100% rename from src/ValueObject/ProductionYear.php rename to app/default/src/ValueObject/ProductionYear.php diff --git a/src/ValueObject/ResultsPerPage.php b/app/default/src/ValueObject/ResultsPerPage.php similarity index 100% rename from src/ValueObject/ResultsPerPage.php rename to app/default/src/ValueObject/ResultsPerPage.php diff --git a/src/ValueObject/SearchResponseInterface.php b/app/default/src/ValueObject/SearchResponseInterface.php similarity index 100% rename from src/ValueObject/SearchResponseInterface.php rename to app/default/src/ValueObject/SearchResponseInterface.php diff --git a/src/ValueObject/SearchResultsInterface.php b/app/default/src/ValueObject/SearchResultsInterface.php similarity index 100% rename from src/ValueObject/SearchResultsInterface.php rename to app/default/src/ValueObject/SearchResultsInterface.php diff --git a/symfony.lock b/app/default/symfony.lock similarity index 100% rename from symfony.lock rename to app/default/symfony.lock diff --git a/templates/base.html.twig b/app/default/templates/base.html.twig similarity index 100% rename from templates/base.html.twig rename to app/default/templates/base.html.twig diff --git a/templates/search/_includes/car.html.twig b/app/default/templates/search/_includes/car.html.twig similarity index 100% rename from templates/search/_includes/car.html.twig rename to app/default/templates/search/_includes/car.html.twig diff --git a/templates/search/_includes/filters.html.twig b/app/default/templates/search/_includes/filters.html.twig similarity index 100% rename from templates/search/_includes/filters.html.twig rename to app/default/templates/search/_includes/filters.html.twig diff --git a/templates/search/_includes/pagination.html.twig b/app/default/templates/search/_includes/pagination.html.twig similarity index 100% rename from templates/search/_includes/pagination.html.twig rename to app/default/templates/search/_includes/pagination.html.twig diff --git a/templates/search/_includes/services.html.twig b/app/default/templates/search/_includes/services.html.twig similarity index 100% rename from templates/search/_includes/services.html.twig rename to app/default/templates/search/_includes/services.html.twig diff --git a/templates/search/emptyList.html.twig b/app/default/templates/search/emptyList.html.twig similarity index 100% rename from templates/search/emptyList.html.twig rename to app/default/templates/search/emptyList.html.twig diff --git a/templates/search/list.html.twig b/app/default/templates/search/list.html.twig similarity index 100% rename from templates/search/list.html.twig rename to app/default/templates/search/list.html.twig diff --git a/docker/service/.env b/docker/service/.env new file mode 100644 index 0000000..7ae8fb6 --- /dev/null +++ b/docker/service/.env @@ -0,0 +1,20 @@ +# name of the project, if blank then docker sets it to a name of a dir that docker-compose.yaml is inside +COMPOSE_PROJECT_NAME=carrental + +# profile to run with docker-compose up (may be multiple separated by comma) makes docker-compose --profile flag not having any effect +# COMPOSE_PROFILES= + +# list of files to combine the total docker-compose.yaml (a kind of include files) +COMPOSE_FILE=_network.yaml:_restart-policy.yaml:_limit-memory.yaml:_service-dependency.yaml:_profiles.yaml:docker-compose.yaml:php-composer-v2.yaml:elasticsearch-v7.yaml:php-fpm-v8.yaml:nginx-v1.yaml + +# application path relative to the docker-compose.yaml file. +app_path=./../../app + +# Path where different service images, their files, Dockerfiles are stored (relative to the docker-compose.yaml) +service_path=. + +# path inside service_path where is Dockerfile and .env files stored +image_path=image + +# path inside service_path where all image files are stored ${service_path}/${image_files_path}/usr/share => /usr/share (path inside a container) +image_files_path=image/files diff --git a/docker/service/.gitignore b/docker/service/.gitignore new file mode 100644 index 0000000..338906f --- /dev/null +++ b/docker/service/.gitignore @@ -0,0 +1 @@ +docker-compose.override.yaml diff --git a/docker/service/README.md b/docker/service/README.md new file mode 100644 index 0000000..7a91a0f --- /dev/null +++ b/docker/service/README.md @@ -0,0 +1,335 @@ +## Table of contents + +- [Project structure](#Project-structure) +- [Projects context](#Projects-context) + - [Specific to this application context](#Specific-to-this-application-context) + - [Generic service module context](#Generic-service-module-context) + - [Copy a service module to any other project](#Copy-a-service-module-to-any-other-project) + - [List of services modules](#List-of-services-modules) + - [Service module volumes](#Service-module-volumes) + - [Service module variables](#Service-module-variables) + - [Service module volume has an application and module context](#Service-module-volume-has-an-application-and-module-context) + - [Merging two contexts](#Merging-two-contexts) +- [Health checks](#Health-checks) + - [Nginx](#Nginx) + - [Elasticsearch](#Elasticsearch) +- [Security](#Security) + - [Permissions](#Permissions) + - [Capabilities](#Capabilities) + - [Network](#Network) + - [Firewall](#Firewall) + - [Vulnerability scan](#Vulnerability-scan) +- [Known Issues](#Known-Issues) +- Typical errors and their reasons + - [Nginx page is loading forever](#Nginx-page-is-loading-forever) + - [Elasticsearch exits with code 125](#Elasticsearch-exits-with-code-125) + + + +## Project structure + +There is a [dirlist.txt](./dirlist.txt) that shows all of the files. + +[^TOC^](#Table-of-contents) + +## Projects context + +There are two contexts in this project: + +- Specific to this application context +- Generic service module context + +[^TOC^](#Table-of-contents) + +### Specific to this application context + +All files that start with underscore + +``` +_limit-memory.yaml +_network.yaml +_profiles.yaml +_restart-policy.yaml +_service-dependency.yaml +``` + +have code related specifically to this application. + +for example [_service-dependency.yaml](_service-dependency.yaml) without comments: + +``` +services: + nginx-v1: + depends_on: + php-fpm-v8: + condition: service_started + elasticsearch-v7: + condition: service_healthy +``` + +defines that Nginx server container should not run unless service `php-fpm-v8` started and service `elasticsearch-v7` started and reported its status as healthy. + +It makes no sense to run Nginx for this application unless it is able to pass requests to the PHP-FPM and make requests to the Elasticsearch service. + +All of the services: `nginx-v1`, `php-fpm-v8` and `elasticsearch-v7` may work independently from each other in a different application. For example + +- `nginx-v1` may serve only static files or work as a load balancer. +- `php-fpm-v8` may respond to other service's requests. +- `elasticsearch-v7` may be used only for Kibana. + +Therefore their dependency is specific for this project and put in a separate file [_service-dependency.yaml](_service-dependency.yaml) instead of putting the `depends_on` key into each of these service's yaml file. + +[^TOC^](#Table-of-contents) + +### Generic service module context + +The key to understand that context is to focus on two words: `generic` and `module`. + +The intention for this context is to be able to copy any service module and with minimal (perfectly none) modification have a base service for other application that may completely differ from this one. + +Every module consist of two things: + +- service directory +- service yaml file named the same as directory. + +#### Copy a service module to any other project + +If you want to copy elasticsearch-v7 module to your application then remember to copy the `elasticsearch-v7` directory and corresponding to it `elasticsearch-v7.yaml` file. + +Currently docker does not support paths relative other than relative to the main docker-compose.yaml file therefore it would make a confusion if `elasticsearch-v7.yaml` file were put inside `elasticsearch-v7` directory and still need to use a path that is a relative to the `docker/service` dir. + +[^TOC^](#Table-of-contents) + +#### List of services modules + +``` +├── [DIR ] elasticsearch-v7 +├── [5.1K] elasticsearch-v7.yaml +├── [DIR ] nginx-v1 +├── [4.4K] nginx-v1.yaml +├── [DIR ] php-composer-v2 +├── [1.2K] php-composer-v2.yaml +├── [DIR ] php-fpm-v8 +├── [ 767] php-fpm-v8.yaml +``` + +There is a naming convention that every service module dir has a suffix of the **major only** version. + +The reason is that for example `elasticsearch-v7` has the suffix `v7` so if currently Elasticsearch is used with the exact version `7.17.1` then having only the major version as a suffix allows you to bump the version number as long it does not brake compatibility and without any need to update dir name. + +An additional benefit of that suffix is that you may have at the same project `elasticsearch-v7` and `elasticsearch-v8` without any conflicts and upgrade or downgrade them as you wish. + +[^TOC^](#Table-of-contents) + +#### Service module volumes + +Example dir tree of `nginx-v1`: + +``` +. +└── [DIR ] image + ├── [ 19] .dockerignore + ├── [ 0] .env.dev + ├── [ 19] .gitignore + ├── [ 602] Dockerfile + └── [DIR ] files + ├── [DIR ] etc + │   └── [DIR ] nginx + │   ├── [DIR ] conf.d + │   │   └── [ 894] default.conf + │   ├── [1007] fastcgi_params + │   ├── [2.8K] koi-utf + │   ├── [2.2K] koi-win + │   ├── [5.1K] mime.types + │   ├── [ 22] modules -> /usr/lib/nginx/modules + │   ├── [1.0K] nginx.conf + │   ├── [ 636] scgi_params + │   ├── [ 664] uwsgi_params + │   └── [3.5K] win-utf + └── [ 166] site.conf +``` + +All image files are stored in the subdir `image`, so you may add other related files like manuals into the dir of the `nginx-v1` service but without confusion if building an image from that service should have them or not. Simply - if files are in the `image` subdir then they are a subject of the build context. + +`image/files` subdir has all of the files that should be either copied to the image by the Dockerfile `COPY` command or they should be mounted by a mount point. Important convention here is that all the files are in the directories that exactly mimic the file structure of the container that will be run from the image you build for example: + +| HOST dir | Container dir | +| -------------------------- | ------------- | +| ./nginx-v1/files/etc/nginx | /etc/nginx | + +Sticking to this convention you will never have any doubts where files are mounted. + +#### Service module variables + +An example [php-fpm-v8.yaml](./php-fpm-v8.yaml) without comments: + +``` +version: "3.7" +services: + php-fpm-v8: + image: ${COMPOSE_PROJECT_NAME}_php:8.1.3-fpm-buster + build: + context: ${service_path:-.}/php-fpm-v8/${image_path} + args: + image: php:8.1.3-fpm-buster + shm_size: '2gb' + target: dev + + env_file: + - "${service_path:-.}/php-fpm-v8/${image_path}/.env.dev" + + user: ${uid:-1000}:${gid:-1000} + + volumes: + - ${app_path:-./app}:/usr/share/nginx/html +``` + +makes as any other service module use of variables: + +- `${COMPOSE_PROJECT_NAME}` that is defined in the [.env](./.env) file and is `carrental` so all service's modules images for this project will have prefix of `carrental_` +- `${service_path:-.}` - defaults to the current directory `docker/service` but may be changed in the [.env](./.env) file. This is the path where docker-compose will look for the dir's of service modules for building them as well as for their dedictated env files. +- `user: ${uid:-1000}:${gid:-1000}` defines the user id and group id of the user that runs the image. For the images that have their own custom Dockerfile like [php-composer-v2/image/Dockerfile](./php-composer-v2/image/Dockerfile) the user id and group id is used for the image build. + +[^TOC^](#Table-of-contents) + +#### Service module volume has an application and module context + +``` + volumes: + - ${app_path:-./app}:/usr/share/nginx/html +``` + +Since they define both source (HOST) and destination (Container) mount points you need to decide if you want to have them inside a service module [php-fpm-v8.yaml](./php-fpm-v8.yaml) or create new application context file for eg. `_volumes.yaml` and put volumes definition there. + +Because some of the services need volumes or bind points for their basic operation (persistent storage for databases) and their paths differ for every service then a decision was made to put volumes inside the service module so if you copy it to another project then you will always have the bare minimum paths on the service side defined and will need to adjust the host side to your needs. That seems to be more reasonable then searching over all of your projects where a particular service was used in order to acquire a knowledge about required paths for its persistence. + +[^TOC^](#Table-of-contents) + +### Merging two contexts + +The [.env](./.env) file has the entry: + +``` +COMPOSE_FILE=_network.yaml:_restart-policy.yaml:_limit-memory.yaml:_service-dependency.yaml:_profiles.yaml:docker-compose.yaml:php-composer-v2.yaml:elasticsearch-v7.yaml:php-fpm-v8.yaml:nginx-v1.yaml +``` + +That has a list of yaml files that are merged when you run for example: + +```sh +docker-compose up +``` + +removing a file from that variable will result in not having it in the merged version that is used by the `docker-compose` to run this app. + +However using [_profiles.yaml](./_profiles.yaml) is much more elegant way to have particular services up. Read about: **Run a specific profile** section of the [../../README.md](../../README.md) + +[^TOC^](#Table-of-contents) + +## Health checks + +They are implemented in the services' Dockerfiles + + +### Nginx +[Dockerfile](./nginx-v1/image/Dockerfile) + +Nginx in the free version don't have any sophisticated health check endpoint therefore a simple one was defined in its [default.conf](./nginx-v1/image/files/etc/nginx/conf.d/default.conf) + +[^TOC^](#Table-of-contents) + +### Elasticsearch + +[Dockerfile](./elasticsearch-v7/image/Dockerfile) + +This sevice has a health check endpoint defined by its vendor. Note that for a single node configuration the health check status as per documentation will never be `green` but at most operable state it will be `yellow`. + +If Elasticsearch is in multi node mode then the health check's status may be `green` or `yellow` depending on the number of nodes that are ok. + +Current health check implementation returns status healthy in case either `green` or `yellow`. + +[^TOC^](#Table-of-contents) + + +## Security + +### Permissions + +Do not run these containers as `root`, there was a significant effort done in order to make them work as a regular user. You may change the user to any and run the `setup.sh` script so it will change the file permissions of all of the files and initialize Elasticsearch db for you. + +However if for any reason you want to run this application as a `root` then all you need is to call `setup.sh` script with `sudo` in front of it. + +[^TOC^](#Table-of-contents) + +### Capabilities + +Look at the [nginx-v1.yaml](#./nginx-v1.yaml) `cap_drop` and `cap_add` sections. + +Do not underestimate the importance of having at least basic security features implemented. + +If you are going do develop your application then play with these values as described in the comments because no one better knows than you if your application is working as expected when restrictions are applied. + +For some of the services adding limitations of the capabilities is a piece of cake. Try it! + +[^TOC^](#Table-of-contents) + +### Network + +For your convenience the `elasticsearch-v7` service is available under the host's `localhost`. When releasing this application to the public domain, remember about disabling that by making the `backend` network internal in [_network.yaml](./_network.yaml) + +[^TOC^](#Table-of-contents) + +#### Firewall + +It would not hurt to use a firewall for the services. You may use UFW or IPTables to limit the exposure of the services since if an image exposes any port it is currently not supported by docker to override that with your own Dockerfile and disable a port exposure. + +[^TOC^](#Table-of-contents) + + +### Vulnerability scan + +It is complety up to you to do it. No scan was made. + +[^TOC^](#Table-of-contents) + +## Known Issues + +`-` or `.` is not allowed for the variable name + +`-` or `.` is not allowed for the variable value, workaround to put value between a single quote `'` or double quote `"` results in passing that variable with the quotes (not just with the content between them). + +[^TOC^](#Table-of-contents) + +## Typical errors and their reasons + +### Nginx page is loading forever + +or log error: + +> nginx: [alert] could not open error log file: open() "/var/log/nginx/error.log" failed (13: Permission denied) + +solution: + +make sure that nginx has at least minimum permissions in its service yaml: + +```yaml +cap_drop: + - ALL + +cap_add: + - CAP_CHOWN + - CAP_DAC_OVERRIDE + - CAP_DAC_READ_SEARCH + - CAP_FOWNER + - CAP_SETGID + - CAP_SETUID +``` + +[^TOC^](#Table-of-contents) + +### Elasticsearch exits with code 125 + +**Solution** + +Elasticsearch don't have enough permissions to access its files. + +[^TOC^](#Table-of-contents) diff --git a/docker/service/_limit-memory.yaml b/docker/service/_limit-memory.yaml new file mode 100644 index 0000000..80a137f --- /dev/null +++ b/docker/service/_limit-memory.yaml @@ -0,0 +1,7 @@ +version: "3.7" + +# memory limits +services: + elasticsearch-v7: + environment: + - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # The ES_JAVA_OPTS variable overrides all other JVM options. We do not recommend using ES_JAVA_OPTS in production. diff --git a/docker/service/_network.yaml b/docker/service/_network.yaml new file mode 100644 index 0000000..3f166a3 --- /dev/null +++ b/docker/service/_network.yaml @@ -0,0 +1,51 @@ +version: "3.7" + +# https://docs.docker.com/compose/compose-file/compose-file-v3/#network-configuration-reference +# https://docs.docker.com/compose/compose-file/compose-file-v3/#internal + +# Note that only nginx-v1 is connected to the frontend and +# backend network. +# All other services are connected to the backend network only. +# Backend network may be set as internal. + +services: + nginx-v1: + ports: + - "9090:80" + networks: + - frontend + - backend + + php-fpm-v8: + environment: + + # Env is here because it is specific to this project but not to the php-fpm service in general. + # also you have here defined networking therefore creation of env that will hold url shows you + # interconnection that happens in the networks. + # + - "DOCKER_ELASTICSEARCH_HOST=http://elasticsearch-v7:9200" + + networks: + - backend + elasticsearch-v7: + # potentially redundant - no need to specify + networks: + - backend + ports: + - 9200:9200 + + php-composer-v2: + environment: + + # Composer needs Elasticsearch host for sending requests by init.sh and request.sh + # + - "DOCKER_ELASTICSEARCH_HOST=http://elasticsearch-v7:9200" + + networks: + - backend + +networks: + frontend: + internal: false + backend: + internal: false # change to true if you don't want that network to be accesible from the HOST diff --git a/docker/service/_profiles.yaml b/docker/service/_profiles.yaml new file mode 100644 index 0000000..e69a6d4 --- /dev/null +++ b/docker/service/_profiles.yaml @@ -0,0 +1,52 @@ +version: "3.7" + +# https://docs.docker.com/compose/compose-file/compose-file-v3/#profiles + +# If you start a single profile for eg. cli +# +# docker-compose --profile cli up --build +# +# but you see that besides cli (composer) many +# other conainers are also runnig then you need to +# stop all of the containers [Ctrl]+[c] or: +# +# docker-compose stop +# +# and run: +# +# docker-compose down --remove-orphans +# +# to remove them before runing that cli profile +# If you run: +# +# docker ps -a +# +# and don't see any containers then running +# +# docker-compose --profile cli up --build +# +# should make only the cli profile (composer) running. + +services: + nginx-v1: + profiles: + - dev + - prod + - web + + php-fpm-v8: + profiles: + - dev + - prod + - web + + elasticsearch-v7: + profiles: + - dev + - prod + - db + + php-composer-v2: + profiles: + - dev + - cli diff --git a/docker/service/_restart-policy.yaml b/docker/service/_restart-policy.yaml new file mode 100644 index 0000000..6f8f138 --- /dev/null +++ b/docker/service/_restart-policy.yaml @@ -0,0 +1,16 @@ +version: "3.7" + +# https://docs.docker.com/compose/compose-file/compose-file-v3/#restart + +services: + nginx-v1: + restart: unless-stopped + + php-fpm-v8: + restart: unless-stopped + + elasticsearch-v7: + restart: unless-stopped + + php-composer-v2: + restart: unless-stopped diff --git a/docker/service/_service-dependency.yaml b/docker/service/_service-dependency.yaml new file mode 100644 index 0000000..13b1434 --- /dev/null +++ b/docker/service/_service-dependency.yaml @@ -0,0 +1,21 @@ +version: "3.7" + +services: + nginx-v1: + depends_on: + + # Service nginx-v1 will be created + # but will be not started unless + # php-fpm-v8 is started and + # elasticsearch-v7 reports healthy status + + # Available conditions: + # service_started + # service_healthy # requies implementation of healthcheck - see Dockerfile of Nginx or Elasticsearch + # service_completed_successfully + + php-fpm-v8: + condition: service_started + + elasticsearch-v7: + condition: service_healthy diff --git a/docker/service/dirlist.txt b/docker/service/dirlist.txt new file mode 100644 index 0000000..2a3e9ef --- /dev/null +++ b/docker/service/dirlist.txt @@ -0,0 +1,107 @@ +. +├── [1.0K] .env +├── [ 28] .gitignore +├── [6.4K] README.md +├── [ 235] _limit-memory.yaml +├── [1.4K] _network.yaml +├── [ 902] _profiles.yaml +├── [ 280] _restart-policy.yaml +├── [ 540] _service-dependency.yaml +├── [ 0] dirlist.txt +├── [ 342] docker-compose.yaml +├── [DIR ] elasticsearch-v7 +│   └── [DIR ] image +│   ├── [ 70] .dockerignore +│   ├── [ 462] .env.dev +│   ├── [ 363] Dockerfile +│   └── [DIR ] files +│   └── [DIR ] usr +│   └── [DIR ] share +│   └── [DIR ] elasticsearch +│   └── [DIR ] config +│   ├── [1.0K] elasticsearch-plugins.example.yml +│   ├── [ 199] elasticsearch.keystore +│   ├── [ 53] elasticsearch.yml +│   ├── [3.2K] jvm.options +│   ├── [DIR ] jvm.options.d +│   ├── [ 19K] log4j2.file.properties +│   ├── [ 10K] log4j2.properties +│   ├── [ 473] role_mapping.yml +│   ├── [ 197] roles.yml +│   ├── [ 0] users +│   └── [ 0] users_roles +├── [5.1K] elasticsearch-v7.yaml +├── [DIR ] nginx-v1 +│   └── [DIR ] image +│   ├── [ 19] .dockerignore +│   ├── [ 0] .env.dev +│   ├── [ 19] .gitignore +│   ├── [ 602] Dockerfile +│   └── [DIR ] files +│   ├── [DIR ] etc +│   │   └── [DIR ] nginx +│   │   ├── [DIR ] conf.d +│   │   │   └── [ 894] default.conf +│   │   ├── [1007] fastcgi_params +│   │   ├── [2.8K] koi-utf +│   │   ├── [2.2K] koi-win +│   │   ├── [5.1K] mime.types +│   │   ├── [ 22] modules -> /usr/lib/nginx/modules +│   │   ├── [1.0K] nginx.conf +│   │   ├── [ 636] scgi_params +│   │   ├── [ 664] uwsgi_params +│   │   └── [3.5K] win-utf +│   └── [ 166] site.conf +├── [4.4K] nginx-v1.yaml +├── [DIR ] php-composer-v2 +│   ├── [ 0] .env.dev +│   └── [DIR ] image +│   └── [ 684] Dockerfile +├── [1.2K] php-composer-v2.yaml +├── [DIR ] php-fpm-v8 +│   └── [DIR ] image +│   ├── [ 28] .dockerignore +│   ├── [ 31] .env.dev +│   ├── [ 28] Dockerfile +│   └── [DIR ] files +│   └── [DIR ] usr +│   └── [DIR ] local +│   └── [DIR ] etc +│   └── [DIR ] php +│   ├── [DIR ] conf.d +│   │   ├── [ 20] docker-php-ext-bcmath.ini +│   │   ├── [ 22] docker-php-ext-calendar.ini +│   │   ├── [ 17] docker-php-ext-dba.ini +│   │   ├── [ 18] docker-php-ext-exif.ini +│   │   ├── [ 16] docker-php-ext-gd.ini +│   │   ├── [ 21] docker-php-ext-gettext.ini +│   │   ├── [ 21] docker-php-ext-imagick.ini +│   │   ├── [ 21] docker-php-ext-mongodb.ini +│   │   ├── [ 20] docker-php-ext-mysqli.ini +│   │   ├── [ 82] docker-php-ext-opcache.ini +│   │   ├── [ 19] docker-php-ext-pcntl.ini +│   │   ├── [ 23] docker-php-ext-pdo_mysql.ini +│   │   ├── [ 23] docker-php-ext-pdo_pgsql.ini +│   │   ├── [ 19] docker-php-ext-pgsql.ini +│   │   ├── [ 20] docker-php-ext-pspell.ini +│   │   ├── [ 19] docker-php-ext-shmop.ini +│   │   ├── [ 18] docker-php-ext-soap.ini +│   │   ├── [ 21] docker-php-ext-sockets.ini +│   │   ├── [ 20] docker-php-ext-sodium.ini +│   │   ├── [ 17] docker-php-ext-svm.ini +│   │   ├── [ 21] docker-php-ext-sysvmsg.ini +│   │   ├── [ 21] docker-php-ext-sysvsem.ini +│   │   ├── [ 21] docker-php-ext-sysvshm.ini +│   │   ├── [ 18] docker-php-ext-tidy.ini +│   │   ├── [ 20] docker-php-ext-trader.ini +│   │   ├── [ 18] docker-php-ext-wddx.ini +│   │   ├── [ 20] docker-php-ext-xmlrpc.ini +│   │   ├── [ 17] docker-php-ext-xsl.ini +│   │   ├── [ 17] docker-php-ext-zip.ini +│   │   └── [ 103] php.ini +│   ├── [ 69K] php.ini-development +│   └── [ 69K] php.ini-production +├── [ 767] php-fpm-v8.yaml +└── [2.5K] setup.sh + +24 directories, 81 files diff --git a/docker/service/docker-compose.yaml b/docker/service/docker-compose.yaml new file mode 100644 index 0000000..23b4bef --- /dev/null +++ b/docker/service/docker-compose.yaml @@ -0,0 +1,18 @@ +version: "3.7" + +# Application specific code is in files that start with an underscore, for exmple: +# +# _network.yaml +# _profiles.yaml + +# Service specific code is in files without underscore, example: +# +# nginx-v1.yaml +# php-fpm-v8.yaml + +# All files are combined by providing their filenames as a value of the +# variable COMPOSE_FILE, for example: +# +# COMPOSE_FILE=_network.yaml:_restart-policy.yaml ... +# +# in the .env file diff --git a/docker/service/elasticsearch-v7.yaml b/docker/service/elasticsearch-v7.yaml new file mode 100644 index 0000000..1dc3217 --- /dev/null +++ b/docker/service/elasticsearch-v7.yaml @@ -0,0 +1,109 @@ +version: "3.7" + +services: + elasticsearch-v7: + + # https://docs.docker.com/compose/reference/envvars/ + + # The image key makes use of the ${COMPOSE_PROJECT_NAME} var + # defined in the .env file so this provides consistent naming + # of the images stored at your machine and allows to list + # only these images that are related to this project by + # docker image ls | grep 'carrental_' + # + image: ${COMPOSE_PROJECT_NAME}_elasticsearch:7.17.1 + + # https://docs.docker.com/compose/compose-file/compose-file-v3/#build + + build: + + # Context is the place where docker-compose may find Dockerfile + # note that there is also .dockerignore that lists paths that + # should not be loaded by the build command into memory when + # building the image from the Dockerfile. + # If there are big files there (logs, database data) + # you may fail building the image because of running out of the RAM + # Sending files to RAM as the build context is only required + # if you want to use these files inside image for eg. by COPY command + # + context: ${service_path:-.}/elasticsearch-v7/${image_path} + args: + + # https://docs.docker.com/engine/reference/builder/#arg + + # The Elasticsearch Dockerfile has ARG image + # that consumes the value from here for the build. + # + image: docker.elastic.co/elasticsearch/elasticsearch:7.17.1@sha256:35f81ab02dda48d6f81ad9be52b25cdbd3a832e8660b48218af805d6de3ddba8 + + shm_size: '2gb' + + # https://docs.docker.com/develop/develop-images/multistage-build/ + + target: dev + + #BUGFIX - docker-compose does not support variables with dot (.) or underscore (_) in name or as a values in the .env files - these values must be defined here + environment: + - discovery.type=single-node + - node.name=elasticsearch + - include_type_name=true + + # Points to blank .env.dev file due to the #BUGFIX as above. + # + env_file: + - "${service_path:-.}/elasticsearch-v7/${image_path}/.env.dev" + + # https://docs.docker.com/compose/compose-file/compose-file-v3/#healthcheck + + # You may override here the healthcheck periods defined in the Dockerfie + # typically it takes around 30 seconds for Elasticsearch to start so + # the start period should be at least that or more in case of using any + # plugins or having a lot of data. + # + # healthcheck: + # start_period: 1m + # interval: 2s + # timeout: 3s + # retries: 3 + + # As per documentation Elastisearch is running by default as user 1000 and group 0 + # You should be fine using your own user id with a fallback to 1000 in case you did not populate uid variable + # but at least the Elasticsearch's user should have group 0 or be added to the 0 group + # Not setting the user:group id here correctly usually results in Elasticserach exiting with the code 125 + # + user: ${uid:-1000}:0 + + #TODO - make your capabilities restrictions for that service + # take nginx-v1 service as an example and check if you may + # limit the capabilities even further. + # who knows, maybe you may grant zero capabilities and this + # service will still work without any problem? + # Take that seriously and look above at the user: ${uid:-1000}:0 + # that means Elasticsearch has the grup id = 0 (admin) + # Should that user be able to use all capabilities? Probably not. + # + # cap_drop: + # - ALL + + # cap_add: + + + # https://docs.docker.com/compose/compose-file/compose-file-v3/#variable-substitution + + # Note the subtitution to dot (.) in case var service_path is not set + # then path will be relative to docker-compose.yaml: ./elasticsearch-v7/... + # instead of absolute: /elasticsearch-v7/... + # Below are short form volumes (bind mounts specifically) that in case of + # missing directory at the HOST machine create that dir with the root access rights. + # Therefore it is important to remove the possibility for docker engine to mess + # with your HOST system files by creating any directories that are starting from the root (/) path. + # With substitution to dot (.) paths will always be relative to the docker-compose.yaml + # unless you set service_path to an absolute path (starting from /) + # You may turn off this auto directory creation by using long form bind mounts + # that access only already existing directories at HOST and don't create new ones. + # Note that data and logs are read and write (:rw) access for the container but config and plugin + # are read only (:ro) because there is no need for the container to save into them but just read them. + # + volumes: + - ${service_path:-.}/elasticsearch-v7/${image_files_path}/usr/share/elasticsearch/data:/usr/share/elasticsearch/data:rw + - ${service_path:-.}/elasticsearch-v7/${image_files_path}/usr/share/elasticsearch/logs/:/usr/share/elasticsearch/logs/:rw diff --git a/docker/service/elasticsearch-v7/image/.dockerignore b/docker/service/elasticsearch-v7/image/.dockerignore new file mode 100644 index 0000000..4d816e4 --- /dev/null +++ b/docker/service/elasticsearch-v7/image/.dockerignore @@ -0,0 +1,2 @@ +files/usr/share/elasticsearch/data +files/usr/share/elasticsearch/logs diff --git a/docker/service/elasticsearch-v7/image/.env.dev b/docker/service/elasticsearch-v7/image/.env.dev new file mode 100644 index 0000000..789eb63 --- /dev/null +++ b/docker/service/elasticsearch-v7/image/.env.dev @@ -0,0 +1,10 @@ +# BUG unexpected character "." in variable name near "discovery.type=\"single-node\"\n# node.name=\"elasticsearch\"\n# include_type_name=\"true\"" +# it worked fine with the old version of docker and docker-compose. +# quoting the variable name did not help +# now moved to the elasticsearch-v7.yaml +# potential solution https://github.com/laradock/laradock/issues/3076#issuecomment-1017484349 + + +# discovery.type="single-node" +# node.name="elasticsearch" +# include_type_name="true" \ No newline at end of file diff --git a/docker/service/elasticsearch-v7/image/Dockerfile b/docker/service/elasticsearch-v7/image/Dockerfile new file mode 100644 index 0000000..17c7384 --- /dev/null +++ b/docker/service/elasticsearch-v7/image/Dockerfile @@ -0,0 +1,12 @@ +ARG image +FROM $image AS dev + +RUN apt-get update && apt-get install -y curl && apt-get clean + +HEALTHCHECK \ +--start-period=60s \ +--interval=2s \ +--retries=3 \ +--timeout=2000ms \ +CMD /bin/sh -c \ +"curl --fail -s http://localhost:9200/_cat/health | cut -d' ' -f4 | grep -qE 'yellow|green' || exit 1" diff --git a/docker/service/nginx-v1.yaml b/docker/service/nginx-v1.yaml new file mode 100644 index 0000000..6a3889b --- /dev/null +++ b/docker/service/nginx-v1.yaml @@ -0,0 +1,117 @@ +version: "3.7" + +services: + nginx-v1: + image: ${COMPOSE_PROJECT_NAME}_nginx:1.17.8 + build: + context: ${service_path:-.}/nginx-v1/${image_path} + args: + image: nginx:1.17.8 + shm_size: '2gb' + target: dev + + # You may override here the health check periods that are + # defined in the Dockerfile by command line switches. + # Usually it takes Nginx only a few seconds to get ready + # so the start_period may be lower than the default 30s + # The timeout is the amount of time to complete the + # url request by the curl command and verify the response. + # Taking into account that the health check request to + # Nginx gets a simple http 200 response with a plain text + # and the request is sent from within the Nginx container + # (by using loopback) then normally Nginx should have no + # problem responding within a few miliseconds therefore + # the timeout around 100 ~ 300 ms is really a long time + # for Nginx to respond if it operates without any problems. + # Nginx does not invole PHP-FPM for the health check response. + # Check the Nginx service Dockerfile for more information + # about the health check. + # + # healthcheck: + # start_period: 10s + # interval: 20ms + # timeout: 100ms + # retries: 3 + + # https://docs.docker.com/compose/compose-file/compose-file-v3/#cap_add-cap_drop + # + # Your service almost never needs all of the capabilities. + # Due to security reasons they should be limited + # to only these ones that are strictly needed + # for the job that the container is intended to do. + # Note that firstly all capabilities are dropped so + # later only a few of them are added. + # by doing that cap_add is esentially a white list + # of the capabilities hence not turing one by a mistake + # will not decrease the security level but only may + # cause the service to not perform its task(s). + # + cap_drop: + - ALL + + # This is a list of all capabilities. + # note that the ones requried by the original image + # are uncommented but also have an additional comment + # this is important so if you start to play with the + # list below and turn on/off some of them by commenting + # or removing the comment at the beginning you will + # be always having the information about these + # capabilites that are bare minimum to the container + # in its base configuration. If you uncomment (add) any cap. + # add to them also a comment about the reason they need + # to be added so later you know why the bare minimum + # capabilities were not enough for your application + # and prehaps turn off (comment) the additional ones + # if project requirements change and the container + # will not no longer require them. + # For your convinience the list has all of the + # capabilities but you may like to have this list + # in your project separately as a reference with + # more verbose comments and just copy and paste + # from that list here, under the cap_add key the + # capabilities you want to add. + # + cap_add: + # - CAP_AUDIT_CONTROL + # - CAP_AUDIT_READ + # - CAP_AUDIT_WRITE + # - CAP_BLOCK_SUSPEND + - CAP_CHOWN # required by the original image + - CAP_DAC_OVERRIDE # required by the original image + - CAP_DAC_READ_SEARCH # required by the original image + - CAP_FOWNER # required by the original image + # - CAP_FSETID + # - CAP_IPC_LOCK + # - CAP_IPC_OWNER + # - CAP_KILL + # - CAP_LEASE + # - CAP_LINUX_IMMUTABLE + # - CAP_MAC_ADMIN + # - CAP_MAC_OVERRIDE + # - CAP_MKNOD + # - CAP_NET_ADMIN + # - CAP_NET_BIND_SERVICE + # - CAP_NET_BROADCAST + # - CAP_NET_RAW + - CAP_SETGID # required by the original image + # - CAP_SETFCAP + # - CAP_SETPCAP + - CAP_SETUID # required by the original image + # - CAP_SYS_ADMIN + # - CAP_SYS_BOOT + # - CAP_SYS_CHROOT + # - CAP_SYS_MODULE + # - CAP_SYS_NICE + # - CAP_SYS_PACCT + # - CAP_SYS_PTRACE + # - CAP_SYS_RAWIO + # - CAP_SYS_RESOURCE + # - CAP_SYS_TIME + # - CAP_SYS_TTY_CONFIG + # - CAP_SYSLOG + # - CAP_WAKE_ALARM + + volumes: + - ${app_path:-./app}:/usr/share/nginx/html:ro + - ${service_path:-.}/nginx-v1/${image_files_path}/var/log:/var/log + - ${service_path:-.}/nginx-v1/${image_files_path}/etc/nginx/conf.d:/etc/nginx/conf.d:ro diff --git a/docker/service/nginx-v1/image/.dockerignore b/docker/service/nginx-v1/image/.dockerignore new file mode 100644 index 0000000..b86eea6 --- /dev/null +++ b/docker/service/nginx-v1/image/.dockerignore @@ -0,0 +1,2 @@ +files/usr +files/var diff --git a/docker/service/nginx-v1/image/.env.dev b/docker/service/nginx-v1/image/.env.dev new file mode 100644 index 0000000..e69de29 diff --git a/docker/service/nginx-v1/image/.gitignore b/docker/service/nginx-v1/image/.gitignore new file mode 100644 index 0000000..b86eea6 --- /dev/null +++ b/docker/service/nginx-v1/image/.gitignore @@ -0,0 +1,2 @@ +files/usr +files/var diff --git a/docker/service/nginx-v1/image/Dockerfile b/docker/service/nginx-v1/image/Dockerfile new file mode 100644 index 0000000..6ec7026 --- /dev/null +++ b/docker/service/nginx-v1/image/Dockerfile @@ -0,0 +1,16 @@ +ARG image +FROM $image AS dev + +RUN apt-get update && apt-get install -y curl && apt-get clean + +# Note that there must be an extra 'health check endpoint' defined for the healthcheck in ./files/etc/nginx/conf.d/default.conf +# or you need to handle that with your PHP application but if you want just to return http response +# then Nginx config file is ideal for that so the health checks don't bother your PHP application. +# +HEALTHCHECK \ +--start-period=30s \ +--interval=2s \ +--retries=3 \ +--timeout=2000ms \ +CMD /bin/sh -c \ +"curl --fail -s http://localhost:80/_cat/health | grep -qE 'green' || exit 1" diff --git a/docker/service/nginx-v1/image/files/etc/nginx/conf.d/default.conf b/docker/service/nginx-v1/image/files/etc/nginx/conf.d/default.conf new file mode 100644 index 0000000..797873f --- /dev/null +++ b/docker/service/nginx-v1/image/files/etc/nginx/conf.d/default.conf @@ -0,0 +1,35 @@ +server { + + listen 80; + client_max_body_size 6M; + root /usr/share/nginx/html/default/public; + + location / { + try_files $uri /index.php$is_args$args; + } + + # health check endpoint + location = /_cat/health { + access_log off; # do not log requests for the health checks + add_header 'Content-Type' 'text/html'; + return 200 "green\n"; # Note the new line terminating the output. + } + + location ~ ^/index\.php(/|$) { + fastcgi_pass php-fpm-v8:9000; + fastcgi_split_path_info ^(.+\.php)(/.*)$; + include fastcgi_params; + fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name; + fastcgi_param DOCUMENT_ROOT $realpath_root; + internal; + fastcgi_param SERVER_PORT $server_port; + } + + location ~ \.php$ { + return 404; + } + + location ~ /\.ht { + deny all; + } +} diff --git a/docker/service/php-composer-v2.yaml b/docker/service/php-composer-v2.yaml new file mode 100644 index 0000000..1d01d9d --- /dev/null +++ b/docker/service/php-composer-v2.yaml @@ -0,0 +1,47 @@ +version: "3.7" + +services: + php-composer-v2: + image: ${COMPOSE_PROJECT_NAME}_composer:2.2.7 + build: + context: ${service_path:-.}/php-composer-v2/${image_path} + args: + image: composer:2.2.7 + uid: ${uid:-1000} + gid: ${gid:-1000} + shm_size: '2gb' + target: dev + + env_file: + - "${service_path:-.}/php-composer-v2/.env.dev" + + user: ${uid:-1000}:${gid:-1000} + + # makes possible to attach to the container shell and perform actions + # for example install dependencies + # + # docker attach carrental-php-composer-v2-1 + # + # once you do what you wanted to do just + # type exit + [enter] to leave the container shell + # + stdin_open: true + tty: true + + #TODO - make your capabilities restrictions for that service + # take nginx-v1 service as an example and check if you may + # limit the capabilities even further. + # Keep in mind that composer may execute installation scripts + # that potentially may be harmful. + # Does composer container really need all capabilities including + # these with the low level kernell commands? No it does not! + + # cap_drop: + # - ALL + + # cap_add: + + working_dir: /app/default + + volumes: + - ${app_path:-./app}:/app diff --git a/docker/service/php-composer-v2/.env.dev b/docker/service/php-composer-v2/.env.dev new file mode 100644 index 0000000..e69de29 diff --git a/docker/service/php-composer-v2/image/Dockerfile b/docker/service/php-composer-v2/image/Dockerfile new file mode 100644 index 0000000..9a3723e --- /dev/null +++ b/docker/service/php-composer-v2/image/Dockerfile @@ -0,0 +1,21 @@ +ARG image +FROM $image AS dev + +ARG user_name=composer +ARG uid=1000 +ARG gid=1000 + +RUN addgroup -g $gid $username $gid || true && \ + adduser -D -h "/home/${user_name}" -u $uid -G $gid $user_name || true && \ + printf "PS1='\u@\h:[\W]\$ '" > "/home/${user_name}/.bashrc" && \ + mkdir -p "/app" "/home/$user_name/.config" "/home/${user_name}/.cache" && \ + chown -R $uid:$gid "/app" && \ + chown -R $uid:$gid "/home/${user_name}/.config" && \ + chown -R $uid:$gid "/home/${user_name}/.cache" + +USER $uid + +# https://docs.docker.com/engine/reference/builder/#cmd +# https://www.gnu.org/software/coreutils/manual/html_node/env-invocation.html#env-invocation + +CMD /usr/bin/env bash diff --git a/docker/service/php-fpm-v8.yaml b/docker/service/php-fpm-v8.yaml new file mode 100644 index 0000000..730d633 --- /dev/null +++ b/docker/service/php-fpm-v8.yaml @@ -0,0 +1,29 @@ +version: "3.7" +services: + php-fpm-v8: + image: ${COMPOSE_PROJECT_NAME}_php:8.1.3-fpm-buster + build: + context: ${service_path:-.}/php-fpm-v8/${image_path} + args: + image: php:8.1.3-fpm-buster + shm_size: '2gb' + target: dev + + env_file: + - "${service_path:-.}/php-fpm-v8/${image_path}/.env.dev" + + user: ${uid:-1000}:${gid:-1000} + + #TODO - make your capabilities restrictions for that service + # take nginx-v1 service as an example and check if you may + # limit the capabilities even further. + # who knows, maybe you may grant zero capabilities and this + # service still will work without any problem? + + # cap_drop: + # - ALL + + # cap_add: + + volumes: + - ${app_path:-./app}:/usr/share/nginx/html diff --git a/docker/service/php-fpm-v8/image/.env.dev b/docker/service/php-fpm-v8/image/.env.dev new file mode 100644 index 0000000..e69de29 diff --git a/docker/service/php-fpm-v8/image/Dockerfile b/docker/service/php-fpm-v8/image/Dockerfile new file mode 100644 index 0000000..0721279 --- /dev/null +++ b/docker/service/php-fpm-v8/image/Dockerfile @@ -0,0 +1,2 @@ +ARG image +FROM $image AS dev diff --git a/docker/service/setup.sh b/docker/service/setup.sh new file mode 100755 index 0000000..7e1a7b5 --- /dev/null +++ b/docker/service/setup.sh @@ -0,0 +1,87 @@ +#!/bin/sh + +set -eu + +export uid=$(id -u) +export gid=$(id -g) + +echo "Running as: $uid:$gid" + +# own all directories and files +sudo chown -R $uid:$gid ../../ + +# add execution for the current user and its group to init scripts +sudo chmod -v ug+x,o-wx \ +../../app/default/.elasticsearch-http-requests/request.sh \ +../../app/default/.elasticsearch-http-requests/init.sh + +# Nginx: create dirs, set owner and group, set access permissions +mkdir -p \ +./nginx-v1/image/files/var/log/nginx && \ +\ +sudo chown -R $uid:$gid \ +./nginx-v1/image/files/var/log/nginx && \ +\ +sudo chmod -v -R ug=rwX,o=rX \ +./nginx-v1/image/files/var/log/nginx && \ + +# Elasticsearch: create dirs, set owner and group, set access permissions +mkdir -p \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/config \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/data \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/logs \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/plugins && \ +\ +sudo chown -R $uid:0 \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/config \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/data \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/logs \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/plugins && \ +\ +sudo chmod -v -R ug=rwX,o=rX \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/config \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/data \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/logs \ +./elasticsearch-v7/image/files/usr/share/elasticsearch/plugins + +# compose up everyting +docker-compose --profile dev up --detach --build --force-recreate + +# call composer install +docker-compose exec php-composer-v2 composer install + +# call init for elasticsearch +docker-compose exec php-composer-v2 /bin/sh -c "/app/default/.elasticsearch-http-requests/init.sh" + +echo "$ docker ps -a" +docker ps +echo "" + +echo "$ docker stats --no-stream" +docker stats --no-stream +echo "" + +echo "$ docker network ls --filter 'type=custom'" +docker network ls --filter 'type=custom' +echo "" + +# inform about opening web browser +echo "Installation complete." +echo "Run web browser and go to:" +echo "" +echo "http://localhost:9090" +echo "" + +echo "To see logs run:" +echo "" +echo "docker-compose logs" +echo "" + +echo "To shut this application down, run:" +echo "" +echo "docker-compose down --remove-orphans" +echo "" + +echo "See the How to play with it? section" +echo "to know other useful commands." +echo ""