What to expect in Lagoon 1.2.0

In anticipation of the Lagoon 1.2.0 release, this blog post is a quick overview of the new features and improvements to expect on December 16th. You can view the milestone over at GitHub, and the high-level product roadmap.

Headline Items

• The first release of Harbor integration to Lagoon

• More notification options (Microsoft Teams and e-mail)

• Improved API and authentication performance

• The first release of amazee.io docker images for PHP 7.4

• Last release of amazee.io docker images for PHP 5.6, 7.0 and 7.1

API and Authentication

This release includes a large amount of work to increase the stability and performance of the Authentication and API subsystems — reducing request volumes, adding sensible retries, and streamlining some permission checks. Furthermore, a number of additions have been made to the API, adding gitURL validation, and a pair of GraphQL queries for `DeleteSshKeyById` and `userIdByEmail`.

Karl Hepworth (@fubarhouse) has contributed an additional task to run `drush core-cron` (usable in the UI, and via GraphQL). This will provide a handy authenticated method to run a cron for users without having to log in to the actual site. Thanks, Karl!

As per any release, a number of underlying versions have been incremented — we now use GraphQL v14 under the hood, the 7.0.1 release of Keycloak, and a number of other Node.js dependencies have been updated.

Build and Deploy

The most significant addition to the Build and Deploy subsystem has been the inclusion of Private Docker Registries. Lagoon now deploys an instance of Harbor, an open source container image registry (goharbor.io), as well as the code needed to fetch a Docker image from a private registry as part of the Lagoon build process.

The addition of Harbor will add another dimension to the current Build and Deploy process, with customers now being able to pre-build their own images (using their own CI/CD tools), and then deploy them inside Openshift. This will provide enhanced predictability as the images deployed to the registry will have been tested throughout the CI process, and improve build speed as Openshift will no longer have to rebuild its images from a Dockerfile, it will consume them from the registry. In addition, Harbor provides integrated vulnerability scanning (via Clair) to further improve the security posture of images used in your projects.

Harbor is still in its early days in Lagoon, but you can expect to hear more from us about it in the coming weeks, along with some examples of how to utilize it.

In addition, the version of the oc binary has (finally!) been updated from 3.7 to 3.11, which has allowed some of the behind-the-scenes improvements in the other areas to go ahead.

Logging and Reporting

The most significant addition to Logging and Reporting is the ability to forward Log notifications to email and Microsoft Teams (in addition to the current Slack and RocketChat endpoints).

There has been a lot of customization work undertaken on individual Lagoon clusters this month to provide better Elasticsearch and Kibana performance — some of this has necessitated Lagoon codebase updates to provide default index patterns, and allow some configuration overrides.

Operators and Provisioning

The majority of the Lagoon clusters in operation support “read-only” access to their databases. Being able to utilize this connection, and not having to share bandwidth with write operations, can bring significant performance increase to some database-intensive operations (dumps, exports, and backups primarily). Whilst the main change has been to the playbook that provisions the databases (https://github.com/amazeeio/dbaas-mariadb-apb), the exposure of a DB_READREPLICA_HOSTS variable will allow individual sites to provision read-replica access as applicable.

Images

The release of Lagoon 1.2.0 brings the first release of PHP7.4-based images, as well as the last PHP5.6, PHP7.0 and PHP7.1 releases, as they are all now officially end-of-life, and out of security support (https://www.php.net/supported-versions.php). In future Lagoon releases, we will not be creating, testing or publishing images for these EOL releases, so the 1.2.0 tag will remain the “latest” version available.

As well as upgrading Drush8 (8.3.1), Composer (1.9.1) and Drupal Console (1.9.4), we have added a couple of additional variables into the PHP image to allow for the customization (and sensible defaults) of max_allowed_packet size (for database connections).

In addition, we will now provide a list of the versions of each major software component per release.

Testing and Local Development

Testing has seen the most significant amount of work ahead of this release. Given the volume of changes that we are currently making to Lagoon, having a fast, efficient, effective and reliable testing process is vital. Whilst a number of these changes have been implemented on our internal testing setup, they are also reusable when developing locally. These changes include utilizing existing Minishift installations (if installed), utilizing KVM on Linux, increasing parallelism, and replacing some of the outdated tests. In addition, we replaced our existing Jenkins setup with a scalable master/slave setup — this has taken the build time for an individual PR down to around an hour, and allows us to build 4 PRs in parallel!

Documentation and Examples / DX

A large part of understanding how to use, develop and contribute to Lagoon rests on the documentation. A large number of enhancements have been made to the documentation for this release — amongst them is a better explanation of the base images that are published alongside Lagoon and a number of code examples.

Automation, Services, and Helpers

Managing a platform like Lagoon, amazee.io has a number of helpful day-to-day tools, scripts, and helpers that perform functions and automate processes. This month has seen improvements to the storage-calculator scripts now that we can utilize the most recent oc binary, and the ability to perform Billing Cost calculations via the API (Thanks and welcome to the team @Justin!)

Admin UI

amazee.io is a part of the larger Amazee Group, and as such, we get the opportunity to work with our colleagues at Amazee Labs regularly. Over the last couple of months, we have been working with John Albin Wilkins to get through some improvements to the Admin UI. The first step in this process has been to implement Storybook (storybook.js.org) to allow us to build the UI components in a more organized manner, with the aim of being able to draw on a broader pool of developers for UI improvement.