WIP

Special thanks to SolomonHD who has created the Traefik setup example to use Terrakube

Original Reference:

https://gist.github.com/SolomonHD/b55be40146b7a53b8f26fe244f5be52e

Local DNS entries

Update the /etc/hosts file adding the following entries:

127.0.0.1 terrakube-api
127.0.0.1 terrakube-ui
127.0.0.1 terrakube-executor
127.0.0.1 terrakube-dex
127.0.0.1 terrakube-registry

Running Terrakube Locally.

git clone https://github.com/AzBuilder/terrakube.git
cd terrakube/docker-compose
docker-compose up -d

Terrakube will be available in the following URL:

• http://terrakube-ui:3000

  • Username: admin@example.com

  • Password: admin

Docker Compose with Open Telemetry Setup

You can also test Terrakube using the Open Telemetry example setup to see how the components are working internally and check some internal logs

git clone https://github.com/AzBuilder/terrakube.git
cd terrakube/telemetry-compose
docker-compose up -d

The docker images are generated using cloud native buildpacks with the maven plugin, more information can be found in these links:

• https://docs.spring.io/spring-boot/docs/current/reference/html/container-images.html#container-images.buildpacks

• https://docs.spring.io/spring-boot/docs/3.1.5/maven-plugin/reference/htmlsingle/#build-image

Alpaquita Linux (Experimental)

From Terrakube 2.17.0 we generate 3 additional docker images for the API, Registry and Executor components that are based on Alpaquita Linux that are designed and optimized for Spring Boot applications.

docker pull azbuilder/api-server:2.17.0-alpaquita
docker pull azbuilder/open-registry:2.17.0-alpaquita
docker pull azbuilder/executor:2.17.0-alpaquita

module "consul" {
  source = "git@github.com:hashicorp/example.git"
}

You can deploy Terrakube to any Kubernetes cluster using the helm chart available in the following repository:

https://github.com/AzBuilder/terrakube-helm-chart

Helm Repository

To use the repository do the following:

helm repo add terrakube-repo https://AzBuilder.github.io/terrakube-helm-chart
helm repo update

General security

Terrakube security is based organizations and groups.

An organization can have one or multiple groups and each group have different kind of access to manage the following options:

• Module

  • Manage terraform modules inside an organization

• VCS

  • Manage private connections to different VCS like Github, Bitbucket, Azure DevOps and Gitlab and handle SSH keys

• Template

  • Manage the custom flows written in Terrakube Configuration Language when running any job inside the platform

• Workspaces

  • Manage the terraform workspaces to run remote terraform operations.

• Providers

  • Manage the terraform providers available inside the platform

Administrator group

May 2024 (2.21.0)

Welcome to the 2.21.0 release from Terrakube! As we reach the midpoint of the year, we’re excited to introduce several new features and enhancements that improve the overall user experience and functionality. Let’s dive into the key highlights:

Terrakube Actions

Terrakube actions allow you to extend the UI in Workspaces, functioning similarly to plugins. These actions enable you to add new functions to your Workspace, transforming it into a hub where you can monitor and manage your infrastructure and gain valuable insights. With Terrakube actions, you can quickly observe your infrastructure and apply necessary actions.

In the below demo, you can view how to use a couple of the new actions to monitor metrics and restart an Azure VM.

Additionally, we are providing an action to interact with OpenAI directly from the workspace. Check out the Terrakube OpenAI Action for more details.

Check our documentation for more details on how to use and start creating Terrakube actions.

Authenticate Providers with Dynamic Credentials

Terrakube's dynamic provider credentials feature allows you to establish a secure trust relationship between Terraform/OpenTofu and your cloud provider. By using unique, short-lived credentials for each job, this feature significantly reduces the risk of compromised credentials. We are excited to announce support for dynamic credentials for the following cloud providers:

• AWS

• Google Cloud Platform

• Azure

This enhancement ensures that your infrastructure management is both secure and efficient, minimizing potential security vulnerabilities while maintaining seamless integration with your cloud environments.

Support for --auto-approve Flag

We are pleased to announce the addition of support for the --auto-approve flag in the CLI-driven workflow. This feature allows for the automatic approval of Terraform and OpenTofu apply operations.

user@pop-os:~/git/simple-terraform$ terraform apply --auto-approve

Running apply in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://8080-azbuilder-terrakube-jdlq1j61x7k.ws-us110.gitpod.io/app/simple/auto-approve/runs/1

Waiting for the plan to start...

***************************************
Running Terraform PLAN
***************************************

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # null_resource.next will be created
  + resource "null_resource" "next" {
      + id = (known after apply)
    }

  # time_sleep.wait_30_seconds will be created
  + resource "time_sleep" "wait_30_seconds" {
      + create_duration = (known after apply)
      + id              = (known after apply)
    }

  # module.time_module.random_integer.time will be created
  + resource "random_integer" "time" {
      + id     = (known after apply)
      + max    = 5
      + min    = 1
      + result = (known after apply)
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + creation_time = (known after apply)
  + fake_data     = {
      + data     = "Hello World"
      + resource = {
          + resource1 = "fake"
        }
    }

module.time_module.random_integer.time: Creating...
module.time_module.random_integer.time: Creation complete after 0s [id=5]
time_sleep.wait_30_seconds: Creating...
time_sleep.wait_30_seconds: Creation complete after 5s [id=2024-04-16T22:25:15Z]
null_resource.next: Creating...
null_resource.next: Creation complete after 0s [id=7860036023219461249]

Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

Outputs:

creation_time = "5s"
fake_data = {
  "data" = "Hello World"
  "resource" = {
    "resource1" = "fake"
  }
}

UI Enhancements

We have made several enhancements to the UI, focusing on improving the overall user experience:

• Buttons: Basic rounded corner adjustments.

• Global Shadow Optimization: Adjusted from three layers of shadows to two layers, used in various components.

• Loading Spinners: Added spinners for loading main pages instead of static loading text.

• Field Validations: Added validations for fields to improve form submission accuracy.

These updates aim to make the Terrakube interface more intuitive and visually appealing.

Bug fixes

We have made some improvements and fixes based on the feedback from the community and the security analysis. You can find the full list of changes for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.21.0

March 2024 (2.20.0)

Welcome to the 2.20.0 release from Terrakube! In our second update of the year, we're excited to share several key highlights with you:

Workspace Importer

Our newly Workspace Importer is designed to improve your migration from Terraform Cloud or Terraform Enterprise to Terrakube. Providing a user-friendly wizard, this feature simplifies the process of transferring all your workspaces to Terrakube, ensuring that variables, tags, state files, and other components are easily copied over. During its beta phase, this feature has been tested with hundreds of workspaces, receiving positive feedback from the community. If you're considering a migration to Terrakube, we highly recommend reviewing our documentation to see how this feature can save you time and effort.

Enhanced API Token Management

We've enhanced the UI for managing API tokens, making it more intuitive and efficient. Now, revoking tokens can be done directly through the UI with ease.

Organization Default Execution Mode Update

This new feature allows you to set the default execution mode—either local or remote—at the organization level.

Enhanced Support for Monorepos in Private Registry

Responding to community feedback regarding the management of Terraform modules within a monorepo, we've made significant enhancements to our Private Registry. Now, you have the capability to specify the path of the module within the monorepo and set a tag prefix when creating a module.

Agents Support

We've expanded Terrakube capabilities with the introduction of multiple terrakube executor agents. Now, you have the flexibility to select the specific agent you wish to use for each workspace. This new feature enables a higher level of execution isolation based on workspaces, offering improved security and customization to suit your project's needs.

Prefix Support for Remote Backend

If you are using the remote backend this enhancement allows you to perform operations across multiple workspaces under the same prefix, improving workspace management and operation execution.

terraform {
  backend "remote" {
    hostname = "8080-azbuilder-terrakube-7tnuq3gnkgf.ws-us108.gitpod.io"
    organization = "simple"

    workspaces {
      prefix = "my-app-"
    }
  }
}

Breaking change for POSTGRESQL user

if you are using POSTGRESQL and using the helm chart 3.16.0 it will require you to add the database port using the property

api.properties.databasePort: "5432"

More information can be found in this issue

Bug fixes

We have made some improvements and fixes based on the feedback from the community and the security analysis. You can find the full list of changes for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.20.0

January 2024 (2.19.0)

As we step into the new year, we are thrilled to announce the release of Terrakube 2.19.0, a significant update that continues to push the boundaries of Infrastructure as Code (IaC) management. Here are some key highlights of this release:

OpenTofu Now Available on Terrakube

With OpenTofu now in General Availability (GA), we've integrated it as an alternative within Terrakube. Now, when you create workspaces, you have the option to select either Terraform or OpenTofu. Terrakube will then tailor the user experience specifically to the tool you choose. Additionally, you can switch your existing workspaces to OpenTofu via the Workspace settings, allowing for a smooth transition between tools.

In the future we are planning to introduce specific features for each Terraform and Opentofu based in their evolution. Also we will introduce support for more Iac tools later.

Enhanced Workspace Overview Page

We've upgraded the Workspace Overview Page to enrich your interaction with your infrastructure. Now, you'll find a detailed count of resources and outputs. Each resource is represented by an icon indicating its type, with current support for AWS and Azure icons. Clicking on a resource name unfolds a detailed view of all its attributes and dependencies, and provides direct access to the resource's documentation.

You can access the attributes and dependencies view from both Workspace Overview Page or using the Visual State.

Webhook Improvements

Now, you have the flexibility to select the default template that will execute for each new push in your Version Control System (VCS) at the workspace level. By default, Terrakube performs Plan and Apply operations, but you can now choose a different template tailored to your specific use case. This enhancement allows for the implementation of customized workflows across different organizations or workspaces.

Additionally, Terrakube now checks for the configured directory in the Workspace and will only run the job when a file changes in the specified directory. This feature is particularly useful for monorepositories, where you want to manage different workspaces from a single repository. This functionality has been implemented for GitLab, GitHub, and Bitbucket.

CLI Driven workflow

When leveraging the CLI-driven workflow, you now have the option to utilize the refresh flag.

Here's an example without the refresh flag:

user@pop-os:~/git/simple-terraform$ terraform plan

Running plan in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://8080-azbuilder-terrakube-3it4oxf0vpt.ws-us107.gitpod.io/app/simple/simple-terraform/runs/2

Waiting for the plan to start...

***************************************
Running Terraform PLAN
***************************************
null_resource.previous: Refreshing state... [id=2942672164070633234]
module.time_module.random_integer.time: Refreshing state... [id=4]
time_sleep.wait_30_seconds: Refreshing state... [id=2023-12-28T20:29:16Z]
null_resource.next: Refreshing state... [id=1447023117451433293]
null_resource.next2: Refreshing state... [id=8910164257527828565]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # null_resource.next3 will be created
  + resource "null_resource" "next3" {
      + id = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Example setting refresh to false:

terraform plan -refresh=false

Running plan in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://8080-azbuilder-terrakube-3it4oxf0vpt.ws-us107.gitpod.io/app/simple/simple-terraform/runs/3

Waiting for the plan to start...

***************************************
Running Terraform PLAN
***************************************

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # null_resource.next3 will be created
  + resource "null_resource" "next3" {
      + id = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

API Token Revocation

You now have the ability to revoke both Personal and Team API Tokens. Currently, this functionality is exclusively accessible via the API, but it will be extended to the UI in a forthcoming release. For more information, please refer to our documentation.

Bug fixes

We have made some improvements and fixes based on the feedback from the community and the security analysis. You can find the full list of changes for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.19.0

December 2023 (2.18.0)

Greetings! As the year comes to a close, we are excited to present you the 2.18.0 release of Terrakube. This version brings you some features and enhancements that we hope you’ll enjoy, some of the key highlights include:

AWS icons in State Diagram

The state diagram now includes all the AWS architecture icons, so you can easily recognize each resource type. This feature will help you visualize your Terraform state in a more intuitive way. We are also working on improving the diagram to group the resources by VPC, location, etc which will give you a closer look at how your architecture is designed using the Terraform state.

Visual State Diagram export option

Following the state diagram enhancements, we have also added the option to save the diagram in different formats. You can export a high-quality image in png, svg or jpeg formats by using the Download and selecting the preferred format. This feature will help you use the diagram for your documentation or presentations.

Self Hosted VCS Providers Support

We have added support for Github Enterprise, GitLab Enterprise Edition and GitLab Community Edition. This means that you can now integrate your workspaces and modules using your self-hosted VCS providers. This is a major milestone in our vision of providing an enterprise-grade experience with Terrakube. In the upcoming releases we will add Bitbucket Server and Azure DevOps Server. For more details in how to use this feature check the following links.

• Github Enterprise

• GitLab Enterprise and Community Editions

Webhooks support for GitLab (Cloud, EE and CE), BitBucket and Github (Cloud and Enterprise)

We have introduced a new feature that will make your workflow more efficient and convenient. Now, Terrakube will automatically trigger a new Plan and Apply job for each new push in the repository that is linked to the workspace. You no longer have to run the jobs manually using the UI or the terraform CLI. This feature will help you keep your infrastructure in sync with your code changes.

Bug fixes

We have made some improvements and fixes based on the feedback from the community and the security analysis. You can find the full list of changes for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.18.0

November 2023 (2.17.0)

Welcome to the 2.17.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Submodules view

You can now view the submodules of your Terraform Module in the open registry. When you select a submodule, you will see its details, such as inputs, outputs and resources.

Workspace Overview

We have enhanced the workspace overview to give you more information at a glance. You can now see the Terraform version, the repository, and the list of resources and outputs for each workspace.

Team API Tokens

You can now generate team API tokens using the Terrakube UI and specify their expiration time in days/minutes/hours. This gives you more flexibility and control over your team’s access to Terrakube. To learn more, please visit our documentation.

Terraform provider for Terrakube

You can use the Terrakube provider for Terraform to manage modules, organizations and so on.

provider "terrakube" {
  endpoint = "http://terrakube-api.minikube.net"
  token    = "(PERSONAL ACCESS TOKEN OR TEAM TOKEN)"
}

data "terrakube_organization" "org" {
  name = "simple"
}

data "terrakube_vcs" "vcs" {
  name            = "sample_vcs"
  organization_id = data.terrakube_organization.org.id
}

data "terrakube_ssh" "ssh" {
  name            = "sample_ssh"
  organization_id = data.terrakube_organization.org.id
}

resource "terrakube_team" "team" {
  name             = "TERRAKUBE_SUPER_ADMIN"
  organization_id  = data.terrakube_vcs.vcs.organization_id
  manage_workspace = false
  manage_module    = false
  manage_provider  = true
  manage_vcs       = true
  manage_template  = true
}

resource "terrakube_module" "module1" {
  name            = "module_public_connection"
  organization_id = data.terrakube_ssh.ssh.organization_id
  description     = "module_public_connection"
  provider_name   = "aws"
  source          = "https://github.com/terraform-aws-modules/terraform-aws-vpc.git"
}

Bug fixes

We’ve addressed some issues reported by the community and fixed some vulnerabilities. You can see the full change log for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.17.0

October 2023 (2.16.0)

Welcome to the 2.16.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Improvement in Workspaces List

We have added more filters to workspaces, so you can easily find the one you need. You can now use the tags or description of the workspaces to narrow down your search. Additionally, you can see the tags of each workspace on its card, without having to enter it. This way, you can quickly identify the workspaces that are relevant to you.

Team API Tokens

In the previous version, you could only generate API tokens that were associated with a user account. Now, you can also generate API tokens that are linked to a specific team. This gives you more flexibility and control over your API access and permissions. To generate a team API token, you need to use the API for now, but we are working on adding this feature to the UI in the next version. You can find more information on how to use team API tokens in our documentation

POST {{terrakubeApi}}/access-token/v1/teams
Authorization: Bearer (PERSONAL ACCESS TOKEN)
Request Body:
{
  "days": "1",
  "group": "TERRAKUBE_ADMIN",
  "description": "Sample Team Token"
}

Bug fixes and Performance

We’ve addressed some issues reported by the community regarding performance,bugs and security. You can see the full change log for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.16.0

August 2023 (2.15.0)

Welcome to the 2.15.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Local Execution Mode

In the previous version, all executions were run remotely in Terrakube. However, starting from this version, you can choose to define the execution mode as either local or remote. By default, all workspaces run remotely. If you want to disable remote execution for a workspace, you can change its execution mode to “Local”. This mode allows you to perform Terraform runs locally using the CLI-driven run workflow.

Accessing State from Other Workspaces

Now its possible to use terraform_remote_state datasource to acces state between workspaces. For further details check our documentation

data "terraform_remote_state" "remote_creation_time" {
  backend = "remote"
  config = {
    organization = "simple"
    hostname = "8080-azbuilder-terrakube-vg8s9w8fhaj.ws-us102.gitpod.io"
    workspaces = {
      name = "simple_tag1"
    }
  }
}

Support Cloud Block with Tags

In our previous version, we introduced support for adding tags to Workspaces. Starting from this version, you can use these tags within the cloud block to apply Terraform operations across multiple workspaces that share the same tags. You can find more details in the CLI-Driven workflow documentation

terraform {
  cloud {
    organization = "simple"
    hostname = "8080-azbuilder-terrakube-md8dylvuzta.ws-us101.gitpod.io"

    workspaces {
      tags = ["networking", "development"]
    }
  }
}

Search modules

We’ve added a search function to the Modules pages, making it easier to find modules.

Bug fixes

We’ve addressed some issues reported by the community, so Terrakube is becoming more stable. You can see the full change log for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.15.0

Thanks for all the community support and contributions. Special thanks to @klinux for implementing the search feature inside Modules.

Jun 2023 (2.14.0)

Welcome to the 2.14.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Terraform cloud block support

This new version lets you use the terraform cloud block to execute remote operations using the CLI driven run workflow.

terraform {
  cloud {
    hostname = "8080-azbuilder-terrakube-q8aleg88vlc.ws-us92.gitpod.io"
    organization = "migrate-org"
    workspaces {
      name = "migrate-state"
    }
  }
}

Enhanced UI error messages for Authorization errors

We have enhanced the UI to handle authorization errors more gracefully.

Workspace Tags

We have enabled tagging functionality within the workspaces. In an upcoming release, you will be able to use these tags in the cloud block feature. For more details please check the terrakube documentation.

Overview Workspace page

With Terrakube’s new workspace Overview page, you can access all the essential information for each of your workspaces in a simple and convenient panel. Initially we are including the last run and workspace tags. We will add more information in upcoming releases.

Improved CLI run workflow logs

From this version onwards, you can view the logs in real time when you use the Terraform CLI remote backend or the cloud block.

SSH key injection

Before, Terrakube only used SSH keys to clone private git repos. Now, Terrakube will inject the keys into the workspace. This way, if your terraform modules use git as source, Terraform will access the modules with the keys. Check the documentation for further details.

For the full changelog for this version please check https://github.com/AzBuilder/terrakube/releases/tag/2.14.0

May 2023 (2.13.0)

Welcome to the 2.13.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Terraform init -migrate-state support

Now you can easily migrate your Terraform state from other tools like TFE or Terraform Cloud to Terrakube. The migration command is supported by using the terraform CLI.

terraform login 8080-azbuilder-terrakube-q8aleg88vlc.ws-us92.gitpod.io
terraform init -migrate-state

Check out our documentation for further details about the process.

Improved Log streaming in the UI

Starting with this version, you can see the logs in the Terrakube UI in almost real time, so you can easily follow up on your Terraform changes.

Also we optimized the code and updated most libs to the latest versions For the full changelog for this version please check https://github.com/AzBuilder/terrakube/releases/tag/2.13.0

Mar 2023 (2.12.0)

Welcome to the 2.12.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Search options in Workspaces

Use the new controls in the workspaces list to filter by status or search by name.

Simplified process to add a new VCS provider

You can now connect to a vcs provider in one step, instead of creating it first and then updating the call back url.

Open Telemetry

We added Open Telemetry to the api, registry and executor components for monitoring Terrakube. In future versions, we will also provide Grafana templates for custom metrics like jobs execution and workspaces usage. Check the documentation for further information.

Improvement to Templates

• In the previous version we introduced the use of importCommands, and now improve reutilization we are adding inputsTerraform and inputsEnv so basically now you can send parameters to your templates in an easy way. For example:

flow:
  - type: "terraformPlan"
    step: 100
    importComands:
      repository: "https://github.com/AzBuilder/terrakube-extensions"
      folder: "templates/terratag"
      branch: "import-template"
      inputsEnv:
        INPUT_IMPORT1: $IMPORT1
        INPUT_IMPORT2: $IMPORT2
      inputsTerraform:
        INPUT_IMPORT1: $IMPORT1
        INPUT_IMPORT2: $IMPORT2
  - type: "terraformApply"
    step: 200

Check the documentation for more details.

• And if you want to create a schedule for a template, you can do it easily now with scheduleTemplates.

flow:
  - type: "customScripts"
    step: 100
    inputsTerraform:
      SIZE: $SIZE_SMALL
    commands:
      - runtime: "BASH"
        priority: 100
        before: true
        script: |
          echo "SIZE : $SIZE"
  - type: "scheduleTemplates"
    step: 200
    templates:
      - name: "schedule1"
        schedule: "0 0/5 * ? * * *"
      - name: "schedule2"
        schedule: "0 0/10 * ? * * *"

Check here for more details.

Restrict Terraform Versions or Provide your own Terraform build

You can now use a custom Terraform CLI build or restrict the Terraform versions in your organization with terrakube. Check the documentation for more details.

Terrakube Installation

We improved the helm chart to make the installation easy and now you can quickly install Terrakube for a Sandbox Test environment in Minikube. If you prefer you can now try Terrakube using Docker compose

Thanks for the community contribution, specially thanks to @Diliz and @jstewart612 for their contributions in the Helm Chart. For the full changelog for this version please check https://github.com/AzBuilder/terrakube/releases/tag/2.12.0

And if you have any idea or suggestion don't hesitate to let us know.

February 2023 (2.11.0)

Welcome to the 2.11.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Adding Global Variables to Organization Settings

Global Variables allow you to define and apply variables one time across all workspaces within an organization. So you can set your cloud provider credentials and reuse it in all the workspaces inside the same organization. See more details in our documentation

Adding CLI and API Driven workflows to workspaces

CLI-driven and API-driven workflows are available now. This mean you can create a workspace in the UI and then connect to the same using the Terraform cli or the Terrakube API. Or if you prefer you can go ahead and connect to your Terrakube organization locally from the Terraform cli. Check the CLI-driven worflow documentation for further details.

For CLI or API driven workspaces you will see the steps directly in the overview tab:

And in the list of workspaces you will see a badge so you can easily identify CLI and API driven workspaces

Improved error handling for invalid templates

If you have an error in your template yaml syntax will be easy to identify it. You will see a more clear message about the job execution failure so you can easily see if the error is related with the execution or with the template configuration.

Import scripts inside the templates

If you have some scripts that you want to reuse, you can import it using the new importCommand this will improve the reuse of templates, so you can share some commons steps with the community in your github repos. See Terrakube docs for more details in how to import scripts

flow:
  - type: "terraformPlan"
    step: 100
    importComands:
      repository: "https://github.com/AzBuilder/terrakube-extensions"
      folder: "templates/terratag"
      branch: "import-template"
  - type: "terraformApply"
    step: 200

Adding UI templates support in workspaces

UI templates extend the idea of templates to the UI, so you can customize how each step will look like, in your custom flow inside Terrakube. Basically you can use standard HTML to build the way in which Terrakube will present the specific step

For example you can present a more user friendly information when extracting data from infracost result. And in the UI if you have a custom Template for that step you will se the Structured UI by default, but you can switch the view to the standard terminal output to see the full logs for that step.

Check some examples:

UI templates will allow to create better UI experiences. And brings new ways to customize the way you use Terrakube in your organization. In the upcoming releases we will create some standards templates that you can reuse for the terraform plan and apply. And also we will provide a Terrakube extensions that allows to create UI templates easily.

Improving organization selection on initial login

When you login to Terrakube for the first time and if you have multiple organizations assigned, then you will see a screen to select your organization.

For the full changelog for this version please check https://github.com/AzBuilder/terrakube/releases/tag/2.11.0

And if you have any idea or suggestion don't hesitate to let us know.

This is the high level architecture of the platform

Component descriptions:

• Terrakube API:

  • Expose a JSON:API or GraphQL API providing endpoints to handle:

    • Organizations.

    • Workspace API with the following support:

      • Terraform state history

      • Terraform output history

      • Terraform Variables (public/secrets)

      • Environment Variables (public/secrets)

      • Cron like support to schedule the jobs execution programmatically

    • Job.

      • Custom terraform flows based on Terrakube Configuration Language

    • Modules

    • Providers

    • Teams

    • Teamplate

• Terrakube Jobs:

  • Automatic process that check for any pending job operations in any workspace to trigger a custom logic flow based on Terrakube Configuration Language

• Terrakube Executor:

  • Service that executes the Terrakube job operations written in Terrakube Configuration Language, handle the terraform state and outputs using cloud storage providers like Azure Storage Account

• Terrakube Open Registry:

  • This component allows to expose an open source private repository protected by Dex that you can use to handle your company private terraform modules or providers.

• Cloud Storage:

  • Cloud storage to handle terraform state, outputs and terraform modules used by terraform CLI

• RDBMS:

  • The platform can be used with any database supported by the Liquibase project.

• Security:

  • All authentication and authorization is handle using Dex.

• Terrakube CLI:

  • Go based CLI that can communicate with the Terrakube API and execute operation for organizations, workspaces, jobs, templates, modules or providers

• Terrakube UI:

  • React based frontend to handle all Terrakube Operations.

The default ingress configuration for terrakube can be customize using a terrakube.yaml like the following:

Now you can install terrakube using the command

Terrakube can be installed in minikube as a sandbox environment with HTTPS, using terrakube with HTTPS will allow to use the Terraform registry and the Terraform remote state backend to be used locally without any issue.

Please follow these instructions:

Requirements:

• Install to generate the local certificates.

We will be using following domains in our test installation:

Generate local CA certificate

Local CA certificate path

Local CA certificate content

Generate certificate for *.minikube.net

These command will generate two files key.pem and cert.pem that we will be using later.

Now we have all the necessary to install Terrakube with HTTPS

Setup Helm Repository

Setup Minikube

Create Kubernetes secret with local certificate

Setup Terrakube namespace

Setup DNS records

Setup Ingress Certificates

Helm Values.

We need to generate a file called value.yaml with the following using the content of our rootCa.pem from the previous step:

Install Terrakube with local HTTPS support

Using Terrakube

The environment has some users, groups and sample data so you can test it quickly.

Visit https://terrakube-ui.minikube.net and login using admin@example.com with password admin

We should be able to use the UI using a valid certificate.

Connect to Terrakube

Using terraform we can connect to the terrakube api and the private registry using the following command with the same credentials that we used to login to the UI.

Terraform Remote State.

Lets create a simple Terraform file.

Execute Terraform Remotely

Terraform Init:

Terraform apply:

Checking the UI will show:

The first step will be to create one azure storage account with the folling containers:

• content

• registry

• tfoutput

• tfstate

Once the storage account is created you will have to get the ""

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

Now you can install terrakube using the command

The first step will be to create one s3 bucket with private access

Once the s3 bucket is created you will need to get the following:

• access key

• secret key

• bucket name

• region

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

Now you can install terrakube using the command:

The easiest way to deploy Terrakube is using our Helm Chart, to learn more about this please review the following:

You can deploy Terrakube using different authentication providers for more information check

Terrakube is an open source collaboration platform for running remote infrastructure as code operations using Terraform or OpenTofu that aims to be a complete replacement for close source tools like Terraform Enterprise, Scalr or Env0.

The platform can be easily installed in any kubernetes cluster, you can easily customize the platform using other open source tools already available for terraform (Example: terratag, infracost, terrascan, etc) and you can integrate the tool with different authentication providers like Azure Active Directory, Google Cloud Identity, Github, Gitlab or any other provider supported by .

Terrakube provides different features like the following:

• Terraform module protocol:

  • This allows Terrakube to expose an open source terraform registry that is protected and is private by default using different Dex for the authentication with different providers.

• Terraform provider protocol:

  • This allows Terrakube to expose an open source terraform registry where you can have create your own private terraform providers or create a mirror of the current terraform providers available in the public Hashicorp registry.

• Handle you infrastructure as code using Organization like closed source tools like Terraform Enterprise and Scalr.

• Easily extended by default integrating with many open source tools available today for terraform CLI.

• Support to run the platform using any RDBMS supported by (mysql, postgres, oracle, sql server, etc).

• Plugins support to extends functionality using different cloud providers and open source tools using the Terrakube Configuration Languague (TCL).

• Easily extends the platform using common technologies like react js, java, spring, elide, liquibase, groovy, bash, go.

• Native integration with the most popular VCS like:

  • Github.

  • Azure DevOps

  • Bitbucket

  • Gitlab

• Support for SSH key (RSA and ED25519)

• Handle Personal Access Token

• Extends and create custom flows when running terraform remote operations with custom logics like:

  • Approvals

  • Budget reviews

  • Security reviews

  • Custom logic that you can build with groovy and bash scripts

The first step will be to create one google storage bucket with private access

Once the google storage bucket is created you will need to get the following:

• project id

• bucket name

• JSON GCP credentials file with access to the storage bucket

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

Now you can install terrakube using the command:

To use a MySQL with your Terrakube deployment create a terrakube.yaml file with the following content:

## Terrakube API properties
api:
  defaultDatabase: false
  loadSampleData: false
  properties:
    databaseType: "MYSQL"
    databaseHostname: "server_name.database.mysql.com"
    databaseName: "database_name"
    databaseUser: "database_user"
    databasePassword: "database_password"

Now you can install terrakube using the command.

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

To use a SQL Azure with your Terrakube deployment create a terrakube.yaml file with the following content:

## Terrakube API properties
api:
  defaultDatabase: false
  loadSampleData: false
  properties:
    databaseType: "SQL_AZURE"
    databaseHostname: "server_name.database.windows.net"
    databaseName: "database_name"
    databaseUser: "database_user"
    databasePassword: "database_password"
    databaseSchema: "dbo"

Now you can install terrakube using the command.

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

The first step will be to deploy a minio instance inside minikube in the terrakube namespace

Create the file minio-setup.yaml that we can use to create the default user and buckets

auth:
  rootUser: "admin"
  rootPassword: "superadmin"
defaultBuckets: "terrakube"

kubectl install --values minio-setup.yaml miniostorage bitnami/minio -n terrakube

Once the minio storage is installed lets get the service name.

kubectl get svc -o wide -n terrakube

The service name for the minio storage should be "miniostorage"

Once minio is installed with a bucket you will need to get the following:

• access key

• secret key

• bucket name

• endpoint (http://miniostorage:9000)

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  minio:
    accessKey: "admin"
    secretKey: "superadmin"
    bucketName: "terrakube"
    endpoint: "http://miniostorage:9000"

Now you can install terrakube using the command:

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

WIP

To use a H2 with your Terrakube deployment create a terrakube.yaml file with the following content:INT

## Terrakube API properties
api:
  defaultDatabase: false
  loadSampleData: true
  properties:
    databaseType: "H2"
    databaseHostname: ""
    databaseName: ""
    databaseUser: ""
    databasePassword: ""

Now you can install terrakube using the command.

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

Terrakube use two secrets internally to sign the personal access token and one internal token for intercomponent comunnication when using the helm deployment you can change this values using the following keys:

security:
  patSecret: "AAAAAAAAAAAAAAAAAAAA"
  internalSecret: "BBBBBBBBBBBBBBBBB"

Terrakube will support terraform cli custom builds or custom cli mirrors, the only requirement is to expose an endpoint with the following structure:

Example Endpoint:

• https://eov1ys4sxa1bfy9.m.pipedream.net/

{
   "name":"terraform",
   "versions":{
      "1.3.9":{
         "builds":[
            {
               "arch":"amd64",
               "filename":"terraform_1.3.9_linux_amd64.zip",
               "name":"terraform",
               "os":"linux",
               "url":"https://releases.hashicorp.com/terraform/1.3.9/terraform_1.3.9_linux_amd64.zip",
               "version":"1.3.9"
            }
         ],
         "name":"terraform",
         "shasums":"terraform_1.3.9_SHA256SUMS",
         "shasums_signature":"terraform_1.3.9_SHA256SUMS.sig",
         "shasums_signatures":[
            "terraform_1.3.9_SHA256SUMS.72D7468F.sig",
            "terraform_1.3.9_SHA256SUMS.sig"
         ],
         "version":"1.3.9"
      }
   }
}

To support custom terrafom cli releases when using the helm chart us the following:

api:
  version: "2.12.0"
  terraformReleasesUrl: "https://eov1ys4sxa1bfy9.m.pipedream.net/"

executor:
  version: "2.12.0"

The following will explain how to run the executor component in"ephemeral" mode.

These environment variables can be used to customize the API component:

• ExecutorEphemeralNamespace (Default value: "terrakube")

• ExecutorEphemeralImage (Defatul value: "azbuilder/executor:2.22.0" )

• ExecutorEphemeralSecret (Default value: "terrakube-executor-secrets" )

The above is basically to control where the job will be created and executed and to mount the secrets required by the executor component

Internally the Executor component will use the following to run in "ephemeral" :

• EphemeralFlagBatch (Default value: "false")

• EphemeralJobData, this contains all the data that the executor need to run.

Requirements

To use Ephemeral executors we need to create the following configuration:

Service Account Creation

apiVersion: v1
kind: ServiceAccount
metadata:
  name: terrakube-api-service-account
  namespace: terrakube

Role Creation

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: terrakube
  name: terrakube-api-role
rules:
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

Role Binding Creation

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: terrakube-api-role-binding
  namespace: terrakube
subjects:
- kind: ServiceAccount
  name: terrakube-api-service-account
  namespace: terrakube
roleRef:
  kind: Role
  name: terrakube-api-role
  apiGroup: rbac.authorization.k8s.io

Helm Chart Configuration

Once the above configuration is created we can deploy the Terrakube API like the following example:

## API properties
api:
  image: "azbuilder/api-server"
  version: "2.22.0"
  serviceAccountName: "terrakube-api-service-account"
  env:
  - name: ExecutorEphemeralNamespace
    value: terrakube
  - name: ExecutorEphemeralImage
    value: azbuilder/executor:2.22.0
  - name: ExecutorEphemeralSecret
    value: terrakube-executor-secrets

Workspace Configuration

Add the environment variable TERRAKUBE_ENABLE_EPHEMERAL_EXECUTOR=1 like the image below

Workspace Execution

Now when the job is running internally Terrakube will create a K8S job and will execute each step of the job in a "ephemeral executor"

Internal Kubernetes Job Example:

Plan Running in a pod:

Apply Running in a different pod:

Node Selector.

If required you can specify the node selector configuration where the pod will be created using something like the following:

api:
  env:
  - name: JAVA_TOOL_OPTIONS
    value: "-Dorg.terrakube.executor.ephemeral.nodeSelector.diskType=ssd -Dorg.terrakube.executor.ephemeral.nodeSelector.nodeType=spot"

The above will be the equivalent to use the Kubernetes YAML like:

  nodeSelector:
    disktype: ssd
    nodeType: spot

Terrakube allow to have one or multiple agents to run jobs, you can have as many agents as you want for a single organization.

To use this feature you could deploy a single executor component using the following values:

# Executor should be enable but we need to customize the apiServiceUrl
executor:
  enabled: true
  apiServiceUrl: "http://terrakube-api-service.terrakube:8080" ## The API is in another namespace called "terrakube"

# We need to disable the default openLdap but we need to provide the internal secret
# so the executor can authenticat with the API and the Registry
security:
  useOpenLDAP: false
  internalSecret: "AxxPdgpCi72f8WhMXCTGhtfMRp6AuBfj"

# We need to disable dex in this deployment
dex:
  enabled: false

# We need to disable default storage MINIO and set some custom values 
# in this example will be deploying like using an external MINIO
#(other backend storage could be used too)
storage:
  defaultStorage: false
  minio:
    accessKey: "admin"
    secretKey: "superadmin"
    bucketName: "terrakube"
    endpoint: "http://terrakube-minio.terrakube:9000" ## MINIO is in another namespace called "terrakube"

# We need to disable API, the default redis and default postgresql database
# But we need to provide some properties like the redis connection
api:
  enabled: false
  defaultRedis: false
  defaultDatabase: false
  properties:
    redisHostname: "terrakube-redis-master.terrakube" ## REDIS is in another namespace called "terrakube"
    redisPassword: "7p9iWVeRV4S944"

# We need to disable registry deployment
registry:
  enabled: false

# We need to disable ui deployment
ui:
  enabled: false

# We need to disable the ingress configuration
# but we need to specify the api and registry URL 
ingress:
  useTls: false
  includeTlsHosts: false
  ui:
    enabled: false
  api:
    enabled: false
    domain: "terrakube-api.minikube.net"
  registry:
    enabled: false
    domain: "terrakube-reg.minikube.net"
  dex:
    enabled: false

Now that we have our values.yaml we can use the following helm command:

helm install --debug --values ./your-values.yaml terrakube terrakube-repo/terrakube -n self-hosted-executor

Now we have a single executor component ready to accept jobs or we could change the number or replicas to have multiple replicas like a pool of agent:

Terrakube components support Open Telemetry by default to enable effective observability.

To enable telemetry inside the Terrakube components please add the following environment variable:

OTEL_JAVAAGENT_ENABLE=true

Once the open telemetry agent is enable we can use other environment variables to setup the monitoring for our application for example to enable jaeger we could add the following using addtional environment variables:

OTEL_TRACES_EXPORTER=jaeger
OTEL_EXPORTER_JAEGER_ENDPOINT=http://jaeger-all-in-one:14250
OTEL_SERVICE_NAME=TERRAKUBE-API

Now we can go the jaeger ui to see if everything is working as expected.

There are several differente configuration options for example:

Jaeger exporter

The Jaeger exporter. This exporter uses gRPC for its communications protocol.

Zipkin exporter

The Zipkin exporter. It sends JSON in Zipkin format to a specified HTTP URL.

Prometheus exporter

The Prometheus exporter.

Open Telemetry Example

One small example to show how to use open telemetry with docker compose can be found in the following URL:

When Terrrakube needs to run behind a corporate proxy the following environment variable can be used in each container:

When using the official helm chart the environment variable setting can be set like the following:

Requirements

• A working Keycloak server with a configured realm.

Steps for configuring Terrakube with Keycloak

For configuring Terrakube with Keycloak using Dex the following steps are involved:

• Keycloak client creation and configuration for Terrakube.

• Configure Terrakube so it works with Keycloak.

• Testing the configuration.

Keycloak client creation

Log in to Keycloak with admin credentials and select Configure > Clients > Create. Define the ID as TerrakubeClient and select openid-connect as its protocol. For Root URL use Terrakube's GUI URL. Then click Save.

A form for finishing the Terrakube client configuration will be displayed. These are the fields that must be fulfilled:

• Name: in this example it has the value of TerrakubeClient.

• Client Protocol: it must be set to openid-connect.

• Access Type: set it to confidential.

• Root URL: Terrakube's UI URL.

• Valid Redirect URIs: set it to *

Then click Save.

Notice that, since we set Access Type to confidential, we have an extra tab titled Credentials. Click the Credentials tab and copy the Secret value. It will be used later when we configure the Terrakube's connector.

Depending on your configuration, Terrakube might expect different client scopes, such as openid, profile, email, groups, etc. You can see if they are assigned to TerrakubeClient by clicking on the Client Scopes tab (in TerrakubeClient).

If they are not assigned, you can assign them by selecting the scopes and clicking on the Add selected button.

If some scope does not exist, you must create it before assigning it to the client. You do this by clicking on Client Scopes, then click on the Create button. This will lead you to a form where you can create the new scope. Then, you can assign it to the client.

Terrakube configuration

We have to configure a Dex connector to use with Keycloak. Add a new connector in Dex's configuration, so it looks like this:

This is the simpler configuration that we can use. Let's see some notes about this fields:

• type: must be oidc (OpenID Connect).

• name: this is the string shown in the connectors list in Terrakube GUI.

• issuer: it refers to the Keycloak server. It has the form [http|https]://<KEYCLOAK_SERVER>/auth/realms/<REALM_NAME>

• clientID: refers to the Client ID configured in Keycloak. They must match.

• clientSecret: must be the secret in the Credentials tab in Keycloak..

• redirectURI: has the form [http|https]://<TERRAKUBE_API>/dex/callback. Notice this is the Terrakube API URL and not the UI URL.

• insecureEnableGroups: this is required to enable groups claims. This way groups defined in Keycloak are brought by Terrakube's Dex connector.

{% hint style="info" %} If your users do not have a name set (First Name field in Keycloak), you must tell oidc which attribute to use as the user's name. You can do this giving the userNameKey:

{% endhint %}

Testing Terrakube authentication

When we click on Terrakube's login button we are given the choice to select the connector we want to use:

Click on Log in with TerrakubeClient. You will be redirected to a login form in Keycloak:

After login, you are redirected back to Terrakube and a dialog asking you to grant access is shown

Click on Grant Access. That is the last step. If everything went right, now you should be logged in Terrakube.

You can handle Terrakube users with differen authentication providers like the following

Requirements

• Google Cloud Identity

• Gooble Storage Bucket

For this example lets image that you will be using the following domains to deploy Terrakube.

• registry.terrakube.gcp.com

• ui.terrakube.gcp.com

• api.terrakube.gcp.com

Setup Google Authentication

You need to complete the Google authentication setup for Dex. You can found information in this

You need to go to your GCP projet and create a new OAuth Application you can follow this steps: firts select "APIs & Services => Credentials"

Once inside the "Credentials" page, you will have to create a new OAuth Client

You can now generate the JSON credentials file for your application, you will use this file later in the helm chart.

Now you can create the DEX configuration, you will use this config later when deploying the helm chart.

The firt step is to clone the repository.

Replace <<CHANGE_THIS>> with the real values, create the values.yaml file and run the helm install

Run the installation

Requirements

• Azure Active Directory

• Azure Storage Account

For this example lets image that you will be using the following domains to deploy Terrakube.

• registry.terrakube.azure.com

• ui.terrakube.azure.com

• api.terrakube.azure.com

Setup Azure Authentication

You need to complete the Azure authentication setup for Dex. You can found information in this

You need to go to your Azure and create a new Application

After the application is created you need to add the redirect URL.

You will also need to add the permission Directory.Read.All and ask a Azure administrator to approve the permission.

Now you can create the DEX configuration, you will use this config later when deploying the helm chart.

The firt step is to clone the repository.

Replace <<CHANGE_THIS>> with the real values, create the values.yaml file and run the helm install

Run the installation

Requirements

• AWS Cognito

• AWS S3 Bucket

For this example lets image that you will be using the following domains to deploy Terrakube.

• registry.terrakube.aws.com

• ui.terrakube.aws.com

• api.terrakube.aws.com

Setup AWS Cognito Authentication

You need to complete the AWS Cognito authentication setup for Dex using the OIDC connector. You can found information in this

You need to create a new Cognito user pool

You can keep the default values

Add the domain name to cognito

Once the user pool is created you will need to create a new application.

Update the application configuration and update the redirect URL configuration.

Now you can create the DEX configuration, you will use this config later when deploying the helm chart.

The firt step is to clone the repository.

Replace <<CHANGE_THIS>> with the real values, create the values.yaml file and run the helm install

Run the installation

Persistent Context is helpfull when you need to save information from the job execution, for example it can be used to save the infracost or save the thread id when using the Slack extension. We can also use it to save any JSON information generated inside Terrakube Jobs.

Using Persistent Contexts with the Terrakube API

In order to save the information the terrakube API exposes the following endpoint

POST {{terrakubeApi}}/context/v1/{{jobId}}
{
  "slackThreadId": "12345667",
  "infracost": {}
}

To get the context you can use the Terrakube API

GET {{terrakubeApi}}/context/v1/{{jobId}}

Using Persistent Contexts in Templates

The persistent context can be used using the Context extension from the Terrakube extension repository. It supports saving a JSON file or saving a new property inside the context JSON.

import Context
        
new Context("$terrakubeApi", "$terrakubeToken", "$jobId", "$workingDirectory").saveFile("infracost", "infracost.json")
new Context("$terrakubeApi", "$terrakubeToken", "$jobId", "$workingDirectory").saveProperty("slackId", "1234567890")

This is an example of a Terrakube template using the persistent job context to save the infracost information.

flow:
- type: "terraformPlan"
  step: 100
  commands:
    - runtime: "GROOVY"
      priority: 100
      after: true
      script: |
        import Infracost

        String credentials = "version: \"0.1\"\n" +
                "api_key: $INFRACOST_KEY \n" +
                "pricing_api_endpoint: https://pricing.api.infracost.io"

        new Infracost().loadTool(
           "$workingDirectory",
           "$bashToolsDirectory", 
           "0.10.12",
           credentials)
        "Infracost Download Completed..."
    - runtime: "BASH"
      priority: 200
      after: true
      script: |
        terraform show -json terraformLibrary.tfPlan > plan.json 
        INFRACOST_ENABLE_DASHBOARD=true infracost breakdown --path plan.json --format json --out-file infracost.json
    - runtime: "GROOVY"
      priority: 300
      after: true
      script: |
        import Context
        
        new Context("$terrakubeApi", "$terrakubeToken", "$jobId", "$workingDirectory").saveFile("infracost", "infracost.json")

        "Save context completed..."