CoLab Kube

Colab Kube is CoLab's custom hosting and devops setup, built on Kubernetes clusters. It enables a consistent deployments and environments, that are scalable and repeatable.

The latest iteration of the CoLab Kube is rolling out in March 2023. CoLab's devops team is revamping the Kube platform with the aim of simplifying and improving the developer experience.

Changes from current Kube platform

One cluster instead of two, with multiple environments per project on the same cluster
Transition to new model of pushing code to a GitLab instance - similar to Pantheon of Platform.sh
New CLI tool to provide easy commands to perform most tasks that developers will need to set up and manage projects on Kube.
New visualization and metrics dashboards to see performance and potential issues with your project on Kube.

Kube cluster setup

The new Kube platform will be changing from having separate clusters for each environment (development, staging, production), to one cluster that will house all of your environments.

We will be leveraging a private GitLab instance to push our code to for deployments, that will include to continuous integration and dedployment tools. You will add this instance as a second git remote on your projects, similar to how one would work with projects hosted on Pantheon or Platform.sh.

We'll also be introducing a new command-line interface (CLI) tool for interacting with the Kube platform, which will include commands for all the common tasks that you would need for connecting a codebase to Kube and managing the Kube environments for your project. Read more on the CLI tool.

Developer Setup for working with CoLab Kube

Here is a quick guide to getting set up to use the CoLab Kube.

1Password (credential / password manager)

CoLab uses the 1Password password manager to store credentials and share them among teams. It is a multiplatform app that works on MacOS, Windows, Linux, iOS and Android. It also has plugins for most major web browsers.

If you don't have 1Password installed on your system, please download and install it.

You should also have received an email at your CoLab email address inviting you to join CoLab Cooperative on 1Password. If you haven't yet accepted, or haven't received an invite, please reach out to the Delivery circle.

Once you have accepted the invitation, you will be able to log in to 1Password with your CoLab email and will be able to access password that have been shared with you, as well as having your own private credential storage.

CoLab Vault

You will receive an email from DevOps with a shared credential for the Vault - where credentials and tokens are stored securely, and then can be passed on as environment variables in your server environment. Please accept this shared credential, and save it in your 1Password app.

The first thing you will do is to log in to the vault using the new credentials you saved in 1Password.

Once logged in you will be brought to the Secrets page.

Vault - Secrets page

You will see one parent entry in the format of user-myname. Click on that user name to see all the available services you have credentials for.

Vault - list of available services

If you click on a service entry, it will bring you to a list of your secrets saved for this service.

Vault secrets - dashboard

You can copy the secret by clicking the clipboard icon, and see it in plain text by clicking the eye icon.

Please test the logins for each service, and take a quick look around to familiarize yourself with each service.

A partial list of services you may have access to:

Kubernetes Dashboard - the Dashboard is a web-based Kubernetes user interface. The Kubernetes Dashboard gives an overview of applications running on your cluster, and provides information on the state of Kubernetes resources in your cluster and on any errors that may have occurred.
GitLab - code repository hosting along with automated deployment workflows. We'll be pushing code to this gitlab instance to trigger deployments to Kube.
MinIO - high-performance, S3 compatible object storage, native to Kubernetes.
Grafana - metrics visualization dashboards.
Kibana - live tailing of application / container logs
pgAdmin - a web-based management tool for Postgres databases.
phpMyAdmin - a web-based administration tool for MySQL and MariaDB databases.

Updated deployment workflow with GitLab instance

As part of the Kube updates, deployments will be triggered via git push to specific branches on a private GitLab instance. This means that the git workflow will change slightly. In addition to your main remote repository on Bitbucket, you will need to add a second remote for your project in git. This may be familiar to those of you working on projects hosted on Pantheon or Platform.sh, as we often use their remotes for deployment only.

To see the remotes you currently have in your code, use:

git remote -v

You’ll probably see a remote called ‘origin’ by default

origin       git@bitbucket.org:colabcoop-ondemand/myproject.git (fetch)
origin       git@bitbucket.org:colabcoop-ondemand/myproject.git (push)

To add a second remote, simply add it with a different name (i.e. ‘gitlab’):

git remote add gitlab git@git.kube.v1.colab.coop/myproject.git

Now when you list your remotes, you will see two:

origin       git@bitbucket.org:colabcoop-ondemand/myproject.git (fetch)
origin       git@bitbucket.org:colabcoop-ondemand/myproject.git (push)
gitlab       git@git.kube.v1.colab.coop/myproject.git (fetch)
gitlab       git@git.kube.v1.colab.coop/myproject.git (push)

Once you have the second remote, you will need to pull from the gitlab remote to get the Kube config - the branch will be the environment name (i.e. development, staging, production, etc.). The following flags will need to be included:

git pull gitlab <branch> --allow-unrelated-histories --no-rebase

Now when you want to push to a Kube environment, simply specify the gitlab remote as the destination:

git push gitlab <branch>

Command-line interface (CLI)

A new command-line interface (CLI) tool will be introduced, with the aim of providing commands for all of the major tasks that a developer will need to interact with the Kube platform. A basic set of commands will be introduced at launch, with more added over time.

The CLI tool will help with tasks such as: * syncing between Kube environments * sync both databases and files that aren't manged with git, such as user-uploaded images etc. * also sync to or from your local development environment and Kube. * interactions with specific tools in the Kube ecosystem

The CLI tool will have built in help on commands by adding a --help flag to each command (you've probably seen this in other CLI tools such as Drush, WP-CLI or Composer)