Lagoon v2.0.0

This release is built on the https://github.com/uselagoon/lagoon-images/releases/tag/21.9.0 images

Security Advisories

There were two security advisories resolved as part of this release

• Lagoon audit logs expose bearer tokens for transactions
  The audit logs in the lagoon-logs message exchange contained the bearer tokens used to authenticate actions undertaken. In this release, the logging output to this exchange has been sanitised to remove these tokens, and also any private keys

• Credentials stored in lagoon API environment variables exposed in logs
  A couple of the stages undertaken in a Lagoon Build could output the contents of Lagoon variables into the build logs, where they are visible to anyone with view access. In this release, these stages now no longer print these variables into the logs. Thanks to @mxr576 for the responsible disclosure.

Please refer to https://github.com/uselagoon/lagoon/blob/main/SECURITY.md for more information on how to responsibly disclose potential security issues.

Other Changes in this release

Custom Backup and Restore location support (#2820) allows projects to define their own S3-compatible buckets for backups and restores - please make sure to read the documentation - as there are some caveats!
Lagoon UI customisation (#2843) allows users to customise how many results are returned in the UI - previously this was limited to 25, but can now be controlled by the user.

Changelog

• Add update_openshift_varchar_length procedure to migration script @shreddedbacon (#2840)
• correctly fix permissions for my_query-cache.cnf @Schnitzel (#2822)
• update project fact filter to query 'like' instead of 'equal to' value @timclifford (#2834)
• Custom Backup and Restore Location Support @cdchris12 (#2820)
• Add a message for truncated results in the UI and allow users to select displayed number @shreddedbacon (#2843)
• Allow pseudo-random cronjobs to schedule on minute 59, and hour 23. @seanhamlin (#2833)
• Upgrade fluentd and enforce Opensearch compatibility in logs-concentrator @smlx (#2821)
• Improve logs-tee performance and documentation @smlx (#2818)

Documentation

• mutation typo @timclifford (#2814)
• Add drupal/purge to suggested Composer command @mglaman (#2849)

Local development and Testing

• Changed tasks retry logic and refresh token permissions @bomoko (#2828)
• reenable SKIP_IMAGE_PUBLISH routine @tobybellwood (#2839)
• Reduce test parallelisation, and increase log debuggability @tobybellwood (#2825)
• add docker_pull routine to makefile @tobybellwood (#2824)

Full Changelog: v2.0.0-rc.9...v2.0.0

lagoon v2.0.0-rc.9

This release is built on the https://github.com/uselagoon/lagoon-images/releases/tag/21.8.0 images

Facts, Tasks, and ProjectRouterPatterns, oh my 😌

Changes in this release

• Deletes from DB rather than marks deleted @bomoko (#2656)
• Adds Redis caching to bb sync script @bomoko (#2802)
• Custom tasks @bomoko (#2779)
• enable 20MB query cache for Keycloak-db @Schnitzel (#2797)
• Update Slack Notification GraphQL to use correct DB table @dan2k3k4 (#2782)
• remove curator, it's not used anymore since we switched to OpenDistro @Schnitzel (#2787)
• remove ELASTICSEARCH_HOST, use ELASTICSEARCH_URL @Schnitzel (#2786)
• parsed user logs and transport to rabbitMQ and Logstash @timclifford (#2800)
• filters out Billing Groups that we don't need to create in Opendistro… @Schnitzel (#2788)
• Fact API changes for fact search @timclifford (#2693)
• Adding check for double dashes in new project names @cdchris12 (#2785)
• Add support for project routerPatterns @shreddedbacon (#2806)
• Update link to amazee.io Kibana in docs @cdchris12 (#2769)

Upgrade instructions:

1. Normal Helm Upgrade
2. connect to api-db container: kubectl exec -it lagoon-core-api-db-0 sh
3. run /rerun_initdb.sh
4. Visit Keycloak UI as admin
5. Navigate to Clients > lagoon-opendistro-security > Mappers > groups
6. Replace script with:

var ArrayList = Java.type("java.util.ArrayList");
var groupsAndRoles = new ArrayList();
var forEach = Array.prototype.forEach;

// add all groups the user is part of
forEach.call(user.getGroups().toArray(), function(group) {
  // remove the group role suffixes
  //lets check if the group has a parent if this is a child
  if(group.getFirstAttribute("type") == "role-subgroup") {
    var parent = group.getParent();
    if(parent.getFirstAttribute("type") == "project-default-group") {
        var projectIds = parent.getFirstAttribute("lagoon-projects");
        if(projectIds !== null) {
            forEach.call(projectIds.split(","), function(g) {
              groupsAndRoles.add("p" + g);  
            });
            return;
        }
    }
  }
 
  var groupName = group.getName().replace(/-owner|-maintainer|-developer|-reporter|-guest/gi,"");
  groupsAndRoles.add(groupName);
  return;
});

// add all roles the user is part of
forEach.call(user.getRoleMappings().toArray(), function(role) {
   var roleName = role.getName();
   groupsAndRoles.add(roleName);
});

exports = groupsAndRoles;

7. connect to api container: kubectl exec deploy/lagoon-core-api -it -- sh
8. run yarn sync:opendistro-security

lagoon - v2.0.0-rc.8

This release is built on the https://github.com/uselagoon/lagoon-images/releases/tag/21.7.0 images

Changes in this release

• Fix some of the residual OpenDistro Group/Tenant issues @tobybellwood (#2778)

Lagoon v2.0.0-rc.7

This release is built on the https://github.com/uselagoon/lagoon-images/releases/tag/21.7.0 images

Changes in this release

• possibility to overwrite kibana index patterns with OVERWRITE_KIBANA_INDEX_PATTERN=true @Schnitzel (#2774)
• small fixes to make tug system lagoon 2 compatible @Schnitzel (#2775)
• new dev command for API @Schnitzel (#2776)
• create lagoon-internal-registry-secret only if does not exist yet @tobybellwood (#2770)
• Add "description" key to OpenDistro-security tenant create call @KorvinSzanto (#2578)
• adds p-N class OpenDistro groups @bomoko (#2768)
• Performance tweaks for Task/Backup UI pages @rocketeerbkw (#2771)

lagoon v2.0.0-rc.6

This release is built on the https://github.com/uselagoon/lagoon-images/releases/tag/21.7.0 images

Changes in this release

• update docker used in Lagoon from 19.03 to 20.10 @tobybellwood (#2705)
• Fix task file uploads properly @rocketeerbkw (#2753)
• update chromatic action node version to 16 @tobybellwood (#2765)
• Set bulk SC for node-persistent pvc in OC template @vincenzodnp (#2760)
• Fixes #2708 - Skip Deploy for Bitbucket @dan2k3k4 (#2709)
• Fixes destructuring of async-retry module @bomoko (#2718)
• Fix logs2rocketchat routes check @shreddedbacon (#2757)
• Performance tweaks for Deployments UI page @rocketeerbkw (#2763)
• load esClient fresh insreat from the clients pool @Schnitzel (#2750)
• Updating k8up template backupcommand and file-extension annotations @cdchris12 (#2745)
• Updating Lagoon Backup docs @cdchris12 (#2756)
• Custom Backup Schedules and Hourly Backup Retention Support @cdchris12 (#2721)

v2.0.0-rc.5

This Lagoon release builds on the https://github.com/uselagoon/lagoon-images/releases/tag/21.6.0 (release) base images

Highlights in this release

A few fixups identified during the Lagoon 1>Lagoon2 upgrade process.

All Changes in this release

• fix wrong usage of object returned by selectProject() @Schnitzel (#2748)
• Change LAGOON_SAFE_PROJECT in LAGOON_PROJECT @vincenzodnp (#2743)
• #2746 fix broken file upload with a workaround: @Schnitzel (#2747)
• use S3_FILES_HOST instead of S3_HOST @Schnitzel (#2742)
• allow OPENSHIFT_REGEX to filter which openshifts are idled @Schnitzel (#2739)
• Update developing Lagoon docs @tobybellwood (#2735)

v2.0.0-rc.4

This Lagoon release builds on the https://github.com/uselagoon/lagoon-images/releases/tag/21.6.0 base images

Highlights in this release

Two minor fixes to the tasks system relating to typecasting, and some more local-dev improvements

All Changes in this release

• Sends task id as string for lagoon-kbd @bomoko (#2733)
• SQL results are not typecast to strings anymore @rocketeerbkw (#2726)
• Don't scan images by default in build/* Makefile targets @smlx (#2732)
• Remove some sleeps from test suite @smlx (#2731)
• Fix duplicate lagoon-tests installed in local test cluster @smlx (#2728)
• Parameterise the kind/push-images Makefile target @smlx (#2729)
• Allow the make jobserver to co-ordinate with sub-makes @smlx (#2727)

v2.0.0-rc.2

This Lagoon release builds on the https://github.com/uselagoon/lagoon-images/releases/tag/21.6.0 base images

Release Higlights

One small fix here to a regression that caused a small bug in the UI. All other changes are local-dev related.

All Changes in this release

• Fix mariasql related regression from #2690 @rocketeerbkw (#2723)
• allow configurable admin creds and port-forward routines to Makefile @tobybellwood (#2710)
• readd install-tests routine @tobybellwood (#2720)
• Split out some more of the features-kubernetes tests @smlx (#2716)
• Simplify retest logic @smlx (#2713)
• Wait for kind control plane @smlx (#2715)
• Avoid helm installing the registry twice in kind/test @smlx (#2719)

v2.0.0-rc.1

This Lagoon release builds on the https://github.com/uselagoon/lagoon-images/releases/tag/21.6.0 base images

We will be adding more detailed information on the various components over at our blog https://dev.to/uselagoon in the coming weeks - keep an eye out!

Release Highlights

• Lagoon has been updated to Node 16, running on Alpine 3.13
• A new notification type to send to a webhook has been added
• There is now an API audit log created for all API actions
• Lagoon now fully supports Harbor v2 as a "problem" source
• The test system has been overhauled to have each set of tests more self-sufficient
• All changes as at Lagoon v1.15.1 have been included

All Changes in this release

• update all images to alpine3.13 @tobybellwood (#2694)
• Move broker services to upstream rabbitmq, deprecate rabbitmq images @tobybellwood (#2571)
• Switch services to Node.js 16 @rocketeerbkw (#2691)
• Use mariadb library instead of mariasql @rocketeerbkw (#2632)
• Adds webhook notification to resolver type @bomoko (#2706)
• Adds first iteration of logs2webhook service @bomoko (#2593)
• Feature/audit logs user activity @timclifford (#2690)
• Parity with Lagoon 1.15.1 release @tobybellwood (#2697)
• Custom ingress template format fix @shreddedbacon (#2677)
• Main Problems system Harbor V2 support @bomoko (#2675)
• Update SSH key-creation-timestamp variable to match the storage structure @jrsouth (#2241)
• Lagoon 2 - Changes permission check logic for AddFacts @bomoko (#2668)
• Removing Billing from UI @justinlevi (#2688)
• Fix active-standby development environment limit check @shreddedbacon (#2682)
• add rabbitmq helmchart @Schnitzel (#2641)
• Split each test into it's own setup/teardown, reduce api-data-watcher-pusher, update tests @tobybellwood (#2653)
• run on lagoon labelled nodes in Jenkins @tobybellwood (#2692)
• add back active-standby tests to controller-kubernetes @tobybellwood (#2704)
• Allow skipping Trivy scanning when building Lagoon @cdchris12 (#2681)
• Bump CI tool versions to latest stable @smlx (#2689)
• Local dev DX improvements and bugfixes @rocketeerbkw (#2643)
• change amazeeio github references to uselagoon @tobybellwood (#2699)
• Adding indents and trying to fix list numbering. @AlannaBurke (#2673)
• Moving content out of docs/docs @AlannaBurke (#2638)

Full diff to Lagoon 1 is v1.15.1...v2.0.0-rc.1

v2.0.0-alpha.9

This Lagoon release builds on the https://github.com/uselagoon/lagoon-images/releases/tag/21.4.0 base images

Release Highlights

• Add Chromatic testing using storybook for UI improvements
• All changes as at Lagoon https://github.com/uselagoon/lagoon/releases/tag/v1.14.5 have been included
• All changes as at Lagoon https://github.com/uselagoon/lagoon/releases/tag/v1.14.4 have been included

Changes in this release

• match lagoon v1.14.5 release @tobybellwood (#2635)
• port all PRs from v1.14.4 release to 2.x @tobybellwood (#2618)
• Feature/storybook on chromatic @timclifford (#2625)
• only run chromatic on UI changes @tobybellwood (#2634)
• fix harborSync (#2283 cherry-pick) @Schnitzel (#2626)
• Updating backupcommands for solr, postgres-single, and mongodb-single @cdchris12 (#2592)