Skip to content

[8.x] Database stored in Docker image
Compare
Choose a tag to compare
@AlexSkrypnyk AlexSkrypnyk released this 30 Apr 23:22
8.x-1.10
c882ad0

This release adds highly demanded and long overdue support for using databases stored in Docker image. Using this workflow allows to build large database images in CI overnight and download databases as a fresh image in the morning to use for development and as a cache in CI.

During development, rebuilding local environments for projects with large databases will no longer be a long process (due to the time it takes to import from the database dump), but rather a matter of restarting a container. On the large databases (>1G) the time is reduced from 15-20 minutes to 15-30 seconds.

Additionally, there is now a support for several database sources: files or docker registry. As a result - there is a support for the following workflows (simplified):

  • Download the database from CURL/FTP/Acquia as a dump file -> Import the database -> Develop/run tests -> Deploy
  • Download the database from CURL/FTP/Acquia as a dump file -> Import into the Docker database image -> Push image to the registry -> Use the image to develop/run tests -> Deploy
  • Download the database from the previous version of the image -> Use the image to develop/run tests -> Deploy

CI configuration now allows to enable using the database cache (file or Docker image) from the builds ran on the previous days. It also now supports nightly builds to cache the database to be used in builds during the day (significant builds speedup).

Lastly, the installer script was rewritten in PHP to allow more robust parameters handling and better development speed. It has own PHPUnit tests and updated Bats integration tests.

And of course, all the new functionality has automated tests.

Docker

  • Updated images to Lagoon v1.4.1.
  • Updated Docker Compose configuration to support custom database image taken from DATABASE_IMAGE environment variable.
  • Removed quite pulling of the images. This is due to requests from developers to see which images were cached and which were downloaded during the build.

CI

  • ❗❗Added support for nightly database builds + test.
  • ❗Added support for per-workflow caching.
  • Added support for cache fallback with a switch - using per-branch build caches from previous days.
  • Increased database job timeout in CI to 30 minutes.

Scripts

  • 🔥🔥🔥 Rewritten installer in PHP.
    • Better installer written in PHP + complies with Drupal coding standards.
    • Installer is interactive by default (previsously was quiet).
    • Added PHPunit tests for installer.
    • Updated existing integration Bats tests for installer.
    • Fixed several issues with automated configuration discovery for existing projects.
    • Installer now always overrides existing files during an update.
  • 🔥🔥🔥 Added support for database-in-Docker-image. This allows to work with DB image with in-built data.
    • ❗❗❗ Use https://github.com/drevops/mariadb-drupal-data as a base image + DB seeding script is included.
    • ❗❗❗ Workflow switch is controlled by DATABASE_IMAGE environment variable (one variable to controll the whole workflow).
    • ❗❗ Added support for exporting image as archive. Used by CI to pass built images between jobs as cache and re-use this cache in the subsequent jobs.
    • ❗ Added support for deploying an image to a registry when exporting (if EXPORT_DB_DEPLOY_DOCKER is set).
    • Added reload-db command. Used to quickly restart database image.
    • Added ahoy pull-db command. Used to pull the latest version of the Database image.
  • Added support for deployment of multiple docker images to Docker registry (according to provided service-to-image map).
  • Added support for archiving and un-archiving of Docker images.
  • Removed DrevOps test files from the downloadable package. This makes the downloadable archive cleaner.
  • Moved DrevOps' tests and documentation under scripts/drevops directory.
  • Added VERBOSE_BUILD variable to show the build progress.
  • Added linting of Bats tests with Shellcheck.
  • Improved bats test helpers and assertions.
  • Added confirmations to ahoy build and ahoy down commands.

Back-end

  • Removed Gherkin patch to run single tests with release of behat/gherkin version 4.6.1.
  • Fixed cron running locally and in CI.

Front-end

No changes

Integrations

  • Added support for ODE environments in Acquia (settings and hooks).
  • Acquia is no longer an integration enabled by default.
  • Lagoon is no longer an integration suggested by default.

Documentation

  • Added more documentation in comments to .env file.
  • Added more FAQs.
  • Added documentation scaffolding (still WIP).
Assets 2
Loading