The docker images are generated using cloud native buildpacks with the maven plugin, more information can be found in these links:

• https://docs.spring.io/spring-boot/docs/current/reference/html/container-images.html#container-images.buildpacks

• https://docs.spring.io/spring-boot/docs/3.1.5/maven-plugin/reference/htmlsingle/#build-image

Alpaquita Linux (Experimental)

From Terrakube 2.17.0 we generate 3 additional docker images for the API, Registry and Executor components that are based on Alpaquita Linux that are designed and optimized for Spring Boot applications.

docker pull azbuilder/api-server:2.17.0-alpaquita
docker pull azbuilder/open-registry:2.17.0-alpaquita
docker pull azbuilder/executor:2.17.0-alpaquita

module "consul" {
  source = "git@github.com:hashicorp/example.git"
}

We use Gitpod to develop the platform. You can have a complete development environment to test all the components and features with one click using the following button.

Running Terrakube in Gitpod is like running the platform in any Kubernetes distribution in Azure, Aws or GCP, so you will be able to use all the features available with one single click without installing any software in your computer and the environment should be ready in only 1 or 2 minute.

Terrakube API uses an in memory database, it will be deleted and preloaded with default information in every startup.

Gitpod Environment Information.

A file called GITPOD.md will have the information for the entire workspace

The configuration is generated dynamically using the following bash script. This is executed by Gitpod in the environment startup process.

./scripts/setupDevelopmentEnvironment.sh

If for some reason we need to execute the script again, it needs to be executed form the /workspace/terrakube folder

The development environment will have the following information:

Start Development Environment

To run all the Terrakube components run the following task:

After a couple of seconds all the components will be running, you can start/stop/restart any component as needed.

Login Development Environment

After the application startup is completed, we can login to the development environment, to get the URL you can execute the following command:

gp url 3000

The Terrakube login will look like this example:

Dex Authentication.

The Gitpod environment is using an OpenLDAP tha is connected to a Dex using the LDAP connector. The OpenLDAP configuration(users and groups) can be found in the following file:

scripts/setup/dex/config-ldap.ldif

The Dex configuration file is generated dynamically and will be available in the following file:

scripts/setup/dex/config-ldap.yaml

The template that is used to generate the Dex configuration file can be found in this path:

scripts/template/dex/template-config-ldap.yaml

Dex login page should look like this example:

Sample Organizations

The development environment will be preloaded with 4 organizations(azure, gcp, aws and simple), this information is loaded inside the API startup and the scripts to load the information can be found in the following xml files

api/src/main/resources/db/changelog/demo-data/aws.xml
api/src/main/resources/db/changelog/demo-data/gcp.xml
api/src/main/resources/db/changelog/demo-data/azure.xml
api/src/main/resources/db/changelog/demo-data/simple.xml

Sample Teams

Depending of the selected organization and user you will see different information available.

Sample Modules

Each organization is preloaded with some modules that can be found in Github like the following:

Templates

Each organization is preloaded with the following templates to run Terrakube jobs:

For more information of how to use templates please refer to the following repository

Workspaces

All the organization have different empty workspaces, the Simple organization can be used for testing because the workspace is using a basic terraform file with the following resources:

• null_resource

• time_sleep

Other workspaces can be created but they will be deleted on each restart unless are added inside the initialization scripts:

api/src/main/resources/db/changelog/demo-data/simple.xml

API Testing

Gitpod has the thunder-client installed to easily test the API without using the UI.

All the environment variables for the collection are generated on each Gitpod workspace startup The environment variables template can be found in the following directory:

scripts/template/thunder-tests/thunderEnvironment.json

To authenticate the thunder client you can use Dex Device Code Authentication using the following request

The response should look like the following and you can use the verification_uri_complete to finish the device authentication.

{
  "device_code": "htlj4w73ftjfplinifqntp7ept6xaratvgauu2nj3nldlcgxjym",
  "user_code": "HLMH-WKXX",
  "verification_uri": "https://5556-azbuilder-terrakube-XXXXX.ws-us54.gitpod.io/dex/device",
  "verification_uri_complete": "https://5556-azbuilder-terrakube-XXXX.ws-us54.gitpod.io/dex/device?user_code=HLMH-WKXX",
  "expires_in": 300,
  "interval": 5
}

Once the Dex device code authentication is completed you can use the following request to get the access token.

Now you can call any method available in the API without the UI

Terraform Login Protocol

Terraform login protocol can be tested using the thunder-client collection:

The response should look similar to the following:

Terraform Module Protocol

There is one example of terraform module protocol inside the thunder-client collection that you can be used for testing purposes executing the following two request.

The response should look like the following:

• Getting versions for the module

• Getting zip file for the module

OpenAPI Spec

The specification can be obtained using the following request:

WIP

Special thanks to SolomonHD who has created the Traefik setup example to use Terrakube

Original Reference:

https://gist.github.com/SolomonHD/b55be40146b7a53b8f26fe244f5be52e

Local DNS entries

Update the /etc/hosts file adding the following entries:

127.0.0.1 terrakube-api
127.0.0.1 terrakube-ui
127.0.0.1 terrakube-executor
127.0.0.1 terrakube-dex
127.0.0.1 terrakube-registry

Running Terrakube Locally.

git clone https://github.com/AzBuilder/terrakube.git
cd terrakube/docker-compose
docker-compose up -d

Terrakube will be available in the following URL:

• http://terrakube-ui:3000

  • Username: admin@example.com

  • Password: admin

Docker Compose with Open Telemetry Setup

You can also test Terrakube using the Open Telemetry example setup to see how the components are working internally and check some internal logs

git clone https://github.com/AzBuilder/terrakube.git
cd terrakube/telemetry-compose
docker-compose up -d

You can deploy Terrakube to any Kubernetes cluster using the helm chart available in the following repository:

Helm Repository

To use the repository do the following:

Terrakube is an open source collaboration platform for running remote infrastructure as code operations using Terraform or OpenTofu that aims to be a complete replacement for close source tools like Terraform Enterprise, Scalr or Env0.

The platform can be easily installed in any kubernetes cluster, you can easily customize the platform using other open source tools already available for terraform (Example: terratag, infracost, terrascan, etc) and you can integrate the tool with different authentication providers like Azure Active Directoy, Google Cloud Identity, Github, Gitlab or any other provider supported by .

Terrakube provides different features like the following:

• Terraform module protocol:

  • This allows Terrakube to expose an open source terraform registry that is protected and is private by default using different Dex for the authentication with different providers.

• Terraform provider protocol:

  • This allows Terrakube to expose an open source terraform registry where you can have create your own private terraform providers or create a mirror of the current terraform providers available in the public Hashicorp registry.

• Handle you infrastructure as code using Organization like closed source tools like Terraform Enterprise and Scalr.

• Easily extended by default integrating with many open source tools available today for terraform CLI.

• Support to run the platform using any RDBMS supported by (mysql, postgres, oracle, sql server, etc).

• Plugins support to extends functionality using different cloud providers and open source tools using the Terrakube Configuration Languague (TCL).

• Easily extends the platform using common technologies like react js, java, spring, elide, liquibase, groovy, bash, go.

• Native integration with the most popular VCS like:

  • Github.

  • Azure DevOps

  • Bitbucket

  • Gitlab

• Support for SSH key (RSA and ED25519)

• Handle Personal Access Token

• Extends and create custom flows when running terraform remote operations with custom logics like:

  • Approvals

  • Budget reviews

  • Security reviews

  • Custom logic that you can build with groovy and bash scripts

Terrakube can be installed in minikube as a sandbox environment to test, to use it please follow this:

Setup Helm Repository

Setup Minikube

Setup Terrakube namespace

Setup DNS records

Install Terrakube

The environment has some users, groups and sample data so you can test it quickly.

Visit http://terrakube-ui.minikube.net and login using admin@example.com with password admin

You can login using the following user and passwords

Select the "simple" organization and the "sample_simple" workspace and run a job.

This is the high level architecture of the platform

Component descriptions:

• Terrakube API:

  • Expose a JSON:API or GraphQL API providing endpoints to handle:

    • Organizations.

    • Workspace API with the following support:

      • Terraform state history

      • Terraform output history

      • Terraform Variables (public/secrets)

      • Environment Variables (public/secrets)

      • Cron like support to schedule the jobs execution programmatically

    • Job.

      • Custom terraform flows based on Terrakube Configuration Language

    • Modules

    • Providers

    • Teams

    • Teamplate

• Terrakube Jobs:

  • Automatic process that check for any pending job operations in any workspace to trigger a custom logic flow based on Terrakube Configuration Language

• Terrakube Executor:

  • Service that executes the Terrakube job operations written in Terrakube Configuration Language, handle the terraform state and outputs using cloud storage providers like Azure Storage Account

• Terrakube Open Registry:

  • This component allows to expose an open source private repository protected by Dex that you can use to handle your company private terraform modules or providers.

• Cloud Storage:

  • Cloud storage to handle terraform state, outputs and terraform modules used by terraform CLI

• RDBMS:

  • The platform can be used with any database supported by the Liquibase project.

• Security:

  • All authentication and authorization is handle using .

• Terrakube CLI:

  • Go based CLI that can communicate with the Terrakube API and execute operation for organizations, workspaces, jobs, templates, modules or providers

• Terrakube UI:

  • React based frontend to handle all Terrakube Operations.

March 2024 (2.20.0)

Welcome to the 2.20.0 release from Terrakube! In our second update of the year, we're excited to share several key highlights with you:

Workspace Importer

Our newly Workspace Importer is designed to improve your migration from Terraform Cloud or Terraform Enterprise to Terrakube. Providing a user-friendly wizard, this feature simplifies the process of transferring all your workspaces to Terrakube, ensuring that variables, tags, state files, and other components are easily copied over. During its beta phase, this feature has been tested with hundreds of workspaces, receiving positive feedback from the community. If you're considering a migration to Terrakube, we highly recommend reviewing to see how this feature can save you time and effort.

Enhanced API Token Management

Organization Default Execution Mode Update

This new feature allows you to set the default execution mode—either local or remote—at the organization level.

Enhanced Support for Monorepos in Private Registry

Responding to community feedback regarding the management of Terraform modules within a monorepo, we've made significant enhancements to our Private Registry. Now, you have the capability to specify the path of the module within the monorepo and set a tag prefix when creating a module.

Agents Support

We've expanded Terrakube capabilities with the introduction of multiple terrakube executor agents. Now, you have the flexibility to select the specific agent you wish to use for each workspace. This new feature enables a higher level of execution isolation based on workspaces, offering improved security and customization to suit your project's needs.

Prefix Support for Remote Backend

If you are using the remote backend this enhancement allows you to perform operations across multiple workspaces under the same prefix, improving workspace management and operation execution.

Breaking change for POSTGRESQL user

if you are using POSTGRESQL and using the helm chart 3.16.0 it will require you to add the database port using the property

Bug fixes

We have made some improvements and fixes based on the feedback from the community and the security analysis. You can find the full list of changes for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.20.0

January 2024 (2.19.0)

As we step into the new year, we are thrilled to announce the release of Terrakube 2.19.0, a significant update that continues to push the boundaries of Infrastructure as Code (IaC) management. Here are some key highlights of this release:

OpenTofu Now Available on Terrakube

With OpenTofu now in General Availability (GA), we've integrated it as an alternative within Terrakube. Now, when you create workspaces, you have the option to select either Terraform or OpenTofu. Terrakube will then tailor the user experience specifically to the tool you choose. Additionally, you can switch your existing workspaces to OpenTofu via the Workspace settings, allowing for a smooth transition between tools.

In the future we are planning to introduce specific features for each Terraform and Opentofu based in their evolution. Also we will introduce support for more Iac tools later.

Enhanced Workspace Overview Page

We've upgraded the Workspace Overview Page to enrich your interaction with your infrastructure. Now, you'll find a detailed count of resources and outputs. Each resource is represented by an icon indicating its type, with current support for AWS and Azure icons. Clicking on a resource name unfolds a detailed view of all its attributes and dependencies, and provides direct access to the resource's documentation.

Webhook Improvements

Now, you have the flexibility to select the default template that will execute for each new push in your Version Control System (VCS) at the workspace level. By default, Terrakube performs Plan and Apply operations, but you can now choose a different template tailored to your specific use case. This enhancement allows for the implementation of customized workflows across different organizations or workspaces.

Additionally, Terrakube now checks for the configured directory in the Workspace and will only run the job when a file changes in the specified directory. This feature is particularly useful for monorepositories, where you want to manage different workspaces from a single repository. This functionality has been implemented for GitLab, GitHub, and Bitbucket.

CLI Driven workflow

Here's an example without the refresh flag:

Example setting refresh to false:

API Token Revocation

Bug fixes

We have made some improvements and fixes based on the feedback from the community and the security analysis. You can find the full list of changes for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.19.0

December 2023 (2.18.0)

Greetings! As the year comes to a close, we are excited to present you the 2.18.0 release of Terrakube. This version brings you some features and enhancements that we hope you’ll enjoy, some of the key highlights include:

AWS icons in State Diagram

The state diagram now includes all the AWS architecture icons, so you can easily recognize each resource type. This feature will help you visualize your Terraform state in a more intuitive way. We are also working on improving the diagram to group the resources by VPC, location, etc which will give you a closer look at how your architecture is designed using the Terraform state.

Visual State Diagram export option

Following the state diagram enhancements, we have also added the option to save the diagram in different formats. You can export a high-quality image in png, svg or jpeg formats by using the Download and selecting the preferred format. This feature will help you use the diagram for your documentation or presentations.

Self Hosted VCS Providers Support

We have added support for Github Enterprise, GitLab Enterprise Edition and GitLab Community Edition. This means that you can now integrate your workspaces and modules using your self-hosted VCS providers. This is a major milestone in our vision of providing an enterprise-grade experience with Terrakube. In the upcoming releases we will add Bitbucket Server and Azure DevOps Server. For more details in how to use this feature check the following links.

Webhooks support for GitLab (Cloud, EE and CE), BitBucket and Github (Cloud and Enterprise)

We have introduced a new feature that will make your workflow more efficient and convenient. Now, Terrakube will automatically trigger a new Plan and Apply job for each new push in the repository that is linked to the workspace. You no longer have to run the jobs manually using the UI or the terraform CLI. This feature will help you keep your infrastructure in sync with your code changes.

Bug fixes

We have made some improvements and fixes based on the feedback from the community and the security analysis. You can find the full list of changes for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.18.0

November 2023 (2.17.0)

Welcome to the 2.17.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Submodules view

You can now view the submodules of your Terraform Module in the open registry. When you select a submodule, you will see its details, such as inputs, outputs and resources.

Workspace Overview

We have enhanced the workspace overview to give you more information at a glance. You can now see the Terraform version, the repository, and the list of resources and outputs for each workspace.

Team API Tokens

Terraform provider for Terrakube

Bug fixes

We’ve addressed some issues reported by the community and fixed some vulnerabilities. You can see the full change log for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.17.0

October 2023 (2.16.0)

Welcome to the 2.16.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Improvement in Workspaces List

We have added more filters to workspaces, so you can easily find the one you need. You can now use the tags or description of the workspaces to narrow down your search. Additionally, you can see the tags of each workspace on its card, without having to enter it. This way, you can quickly identify the workspaces that are relevant to you.

Team API Tokens

Bug fixes and Performance

We’ve addressed some issues reported by the community regarding performance,bugs and security. You can see the full change log for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.16.0

August 2023 (2.15.0)

Welcome to the 2.15.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Local Execution Mode

In the previous version, all executions were run remotely in Terrakube. However, starting from this version, you can choose to define the execution mode as either local or remote. By default, all workspaces run remotely. If you want to disable remote execution for a workspace, you can change its execution mode to “Local”. This mode allows you to perform Terraform runs locally using the CLI-driven run workflow.

Accessing State from Other Workspaces

Support Cloud Block with Tags

Search modules

We’ve added a search function to the Modules pages, making it easier to find modules.

Bug fixes

We’ve addressed some issues reported by the community, so Terrakube is becoming more stable. You can see the full change log for this version here https://github.com/AzBuilder/terrakube/releases/tag/2.15.0

Thanks for all the community support and contributions. Special thanks to @klinux for implementing the search feature inside Modules.

Jun 2023 (2.14.0)

Welcome to the 2.14.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Terraform cloud block support

Enhanced UI error messages for Authorization errors

We have enhanced the UI to handle authorization errors more gracefully.

Workspace Tags

Overview Workspace page

With Terrakube’s new workspace Overview page, you can access all the essential information for each of your workspaces in a simple and convenient panel. Initially we are including the last run and workspace tags. We will add more information in upcoming releases.

Improved CLI run workflow logs

From this version onwards, you can view the logs in real time when you use the Terraform CLI remote backend or the cloud block.

SSH key injection

For the full changelog for this version please check https://github.com/AzBuilder/terrakube/releases/tag/2.14.0

May 2023 (2.13.0)

Welcome to the 2.13.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Terraform init -migrate-state support

Now you can easily migrate your Terraform state from other tools like TFE or Terraform Cloud to Terrakube. The migration command is supported by using the terraform CLI.

Improved Log streaming in the UI

Starting with this version, you can see the logs in the Terrakube UI in almost real time, so you can easily follow up on your Terraform changes.

Also we optimized the code and updated most libs to the latest versions For the full changelog for this version please check https://github.com/AzBuilder/terrakube/releases/tag/2.13.0

Mar 2023 (2.12.0)

Welcome to the 2.12.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Search options in Workspaces

Use the new controls in the workspaces list to filter by status or search by name.

Simplified process to add a new VCS provider

You can now connect to a vcs provider in one step, instead of creating it first and then updating the call back url.

Open Telemetry

Improvement to Templates

• In the previous version we introduced the use of importCommands, and now improve reutilization we are adding inputsTerraform and inputsEnv so basically now you can send parameters to your templates in an easy way. For example:

• And if you want to create a schedule for a template, you can do it easily now with scheduleTemplates.

Restrict Terraform Versions or Provide your own Terraform build

Terrakube Installation

Thanks for the community contribution, specially thanks to @Diliz and @jstewart612 for their contributions in the Helm Chart. For the full changelog for this version please check https://github.com/AzBuilder/terrakube/releases/tag/2.12.0

And if you have any idea or suggestion don't hesitate to let us know.

February 2023 (2.11.0)

Welcome to the 2.11.0 release of Terrakube. There are a couple of updates in this version that we hope you'll like, some of the key highlights include:

Adding Global Variables to Organization Settings

Adding CLI and API Driven workflows to workspaces

For CLI or API driven workspaces you will see the steps directly in the overview tab:

And in the list of workspaces you will see a badge so you can easily identify CLI and API driven workspaces

Improved error handling for invalid templates

If you have an error in your template yaml syntax will be easy to identify it. You will see a more clear message about the job execution failure so you can easily see if the error is related with the execution or with the template configuration.

Import scripts inside the templates

Adding UI templates support in workspaces

UI templates extend the idea of templates to the UI, so you can customize how each step will look like, in your custom flow inside Terrakube. Basically you can use standard HTML to build the way in which Terrakube will present the specific step

For example you can present a more user friendly information when extracting data from infracost result. And in the UI if you have a custom Template for that step you will se the Structured UI by default, but you can switch the view to the standard terminal output to see the full logs for that step.

Check some examples:

UI templates will allow to create better UI experiences. And brings new ways to customize the way you use Terrakube in your organization. In the upcoming releases we will create some standards templates that you can reuse for the terraform plan and apply. And also we will provide a Terrakube extensions that allows to create UI templates easily.

Improving organization selection on initial login

When you login to Terrakube for the first time and if you have multiple organizations assigned, then you will see a screen to select your organization.

For the full changelog for this version please check https://github.com/AzBuilder/terrakube/releases/tag/2.11.0

And if you have any idea or suggestion don't hesitate to let us know.

The first step will be to create one google storage bucket with private access

Once the google storage bucket is created you will need to get the following:

• project id

• bucket name

• JSON GCP credentials file with access to the storage bucket

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  gcp:
    projectId: "sample project"
    bucketName: "sampledata"
    credentials: |
      {
        "type": "service_account",
        "project_id": "XXXXXXX",
        ......
      }

Now you can install terrakube using the command:

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

The first step will be to create one azure storage account with the folling containers:

• content

• registry

• tfoutput

• tfstate

Once the storage account is created you will have to get the "Access Key"

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  azure:
    storageAccountName: "<<STORAGE ACCOUNT NAME>>"
    storageAccountResourceGroup: "<< RESOURCE GROUP >>"
    storageAccountAccessKey: "<< STORAGE ACCOUNT ACCESS KEY"

Now you can install terrakube using the command

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

The first step will be to create one s3 bucket with private access

Once the s3 bucket is created you will need to get the following:

• access key

• secret key

• bucket name

• region

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  aws:
    accessKey: "rqerqw"
    secretKey: "sadfasfdq"
    bucketName: "qerqw"
    region: "us-east-1"

Now you can install terrakube using the command:

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

To authenticate users Terrakube implement DEX so you can authenticate using differente providers using dex connectors like the following:

• Azure Active Direcory

• Google Cloud Identity

• Amazon Cognito

• Github Authentication

• Gitlab Authentictaion

• OIDC

• LDAP

• etc.

The Terrakube helm chart is using DEX as dependency, so you can quickly implement it using any exiting dex configuration.

To customize the DEX setup just create a simple terrakube.yaml and update the configuration like the following example:

# UPDATE THE DEX CLIENT ID AND SCOPE 
security:
  dexClientId: "example-app"
  dexClientScope: "email openid profile offline_access groups"
  useOpenLDAP: false
  
# UPDATE THE DEX CONFIG
dex:
  config:
    issuer: http://terrakube-api.minikube.net/dex

    storage:
      type: memory
    web:
      http: 0.0.0.0:5556
      allowedOrigins: ['*']
      skipApprovalScreen: true
    oauth2:
      responseTypes: ["code", "token", "id_token"] 

    connectors:
    - type: ldap
      name: OpenLDAP
      id: ldap
      config:
        # The following configurations seem to work with OpenLDAP:
        #
        # 1) Plain LDAP, without TLS:
        host: terrakube-openldap-service:389
        insecureNoSSL: true
        #
        # 2) LDAPS without certificate validation:
        #host: localhost:636
        #insecureNoSSL: false
        #insecureSkipVerify: true
        #
        # 3) LDAPS with certificate validation:
        #host: YOUR-HOSTNAME:636
        #insecureNoSSL: false
        #insecureSkipVerify: false
        #rootCAData: 'CERT'
        # ...where CERT="$( base64 -w 0 your-cert.crt )"

        # This would normally be a read-only user.
        bindDN: cn=admin,dc=example,dc=org
        bindPW: admin

        usernamePrompt: Email Address

        userSearch:
          baseDN: ou=People,dc=example,dc=org
          filter: "(objectClass=person)"
          username: mail
          # "DN" (case sensitive) is a special attribute name. It indicates that
          # this value should be taken from the entity's DN not an attribute on
          # the entity.
          idAttr: DN
          emailAttr: mail
          nameAttr: cn

        groupSearch:
          baseDN: ou=Groups,dc=example,dc=org
          filter: "(objectClass=groupOfNames)"

          userMatchers:
            # A user is a member of a group when their DN matches
            # the value of a "member" attribute on the group entity.
          - userAttr: DN
            groupAttr: member

          # The group name should be the "cn" value.
          nameAttr: cn

    staticClients:
    - id: example-app
      redirectURIs:
      - 'http://terrakube-ui.minikube.net'
      - '/device/callback'
      - 'http://localhost:10000/login'
      - 'http://localhost:10001/login'
      name: 'example-app'
      public: true

The first step will be to deploy a minio instance inside minikube in the terrakube namespace

Create the file minio-setup.yaml that we can use to create the default user and buckets

auth:
  rootUser: "admin"
  rootPassword: "superadmin"
defaultBuckets: "terrakube"

kubectl install --values minio-setup.yaml miniostorage bitnami/minio -n terrakube

Once the minio storage is installed lets get the service name.

kubectl get svc -o wide -n terrakube

The service name for the minio storage should be "miniostorage"

Once minio is installed with a bucket you will need to get the following:

• access key

• secret key

• bucket name

• endpoint (http://miniostorage:9000)

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  minio:
    accessKey: "admin"
    secretKey: "superadmin"
    bucketName: "terrakube"
    endpoint: "http://miniostorage:9000"

Now you can install terrakube using the command:

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

To use a MySQL with your Terrakube deployment create a terrakube.yaml file with the following content:

## Terrakube API properties
api:
  defaultDatabase: false
  loadSampleData: false
  properties:
    databaseType: "MYSQL"
    databaseHostname: "server_name.database.mysql.com"
    databaseName: "database_name"
    databaseUser: "database_user"
    databasePassword: "database_password"

Now you can install terrakube using the command.

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

WIP

Terrakube can be installed in minikube as a sandbox environment with HTTPS, using terrakube with HTTPS will allow to use the Terraform registry and the Terraform remote state backend to be used locally without any issue.

Please follow these instructions:

Requirements:

• Install to generate the local certificates.

We will be using following domains in our test installation:

Generate local CA certificate

Local CA certificate path

Local CA certificate content

Generate certificate for *.minikube.net

These command will generate two files key.pem and cert.pem that we will be using later.

Now we have all the necessary to install Terrakube with HTTPS

Setup Helm Repository

Setup Minikube

Create Kubernetes secret with local certificate

Setup Terrakube namespace

Setup DNS records

Setup Ingress Certificates

Helm Values.

We need to generate a file called value.yaml with the following using the content of our rootCa.pem from the previous step:

Install Terrakube with local HTTPS support

Using Terrakube

The environment has some users, groups and sample data so you can test it quickly.

Visit https://terrakube-ui.minikube.net and login using admin@example.com with password admin

We should be able to use the UI using a valid certificate.

Connect to Terrakube

Using terraform we can connect to the terrakube api and the private registry using the following command with the same credentials that we used to login to the UI.

Terraform Remote State.

Lets create a simple Terraform file.

Execute Terraform Remotely

Terraform Init:

Terraform apply:

Checking the UI will show:

Adding certificates at runtime

Terrakube componentes (api, registry and executor) are using to create the docker images

When using buildpack to add a custom CA certificate at runtime you need to do the following:

Provide the following environment variable to the container:

Inside the path there is a folder call "ca-certificates"

We need to mount some information to that path

Inside this folder we should put out custom PEM CA certs and one additional file call type

The content of the file type is just the text "ca-certificates"

Finally your helm terrakube.yaml should look something like this because we are mounting out CA certs and the file called type in the following path " /mnt/platform/bindings/ca-certificates"

Checking the terrakube component two additional ca certs are added inside the sytem truststore

Additinal information about buildpacks can be found in this link:

Adding certificate at build time

Terrakube allow to add the certs when building the application, to use this option use the following:

The certs will be added at runtime as the following image.

To use a PostgreSQL with your Terrakube deployment create a terrakube.yaml file with the following content:

## Terrakube API properties
api:
  defaultDatabase: false
  loadSampleData: false
  properties:
    databaseType: "POSTGRESQL"
    databaseHostname: "server_name.postgres.database.com"
    databaseName: "database_name"
    databaseUser: "database_user"
    databasePassword: "database_password"
    databaseSslMode: "disable"
    databasePort: "5432"

Now you can install terrakube using the command.

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

To use a SQL Azure with your Terrakube deployment create a terrakube.yaml file with the following content:

## Terrakube API properties
api:
  defaultDatabase: false
  loadSampleData: false
  properties:
    databaseType: "SQL_AZURE"
    databaseHostname: "server_name.database.windows.net"
    databaseName: "database_name"
    databaseUser: "database_user"
    databasePassword: "database_password"
    databaseSchema: "dbo"

Now you can install terrakube using the command.

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

Terrakube use two secrets internally to sign the personal access token and one internal token for intercomponent comunnication when using the helm deployment you can change this values using the following keys:

security:
  patSecret: "AAAAAAAAAAAAAAAAAAAA"
  internalSecret: "BBBBBBBBBBBBBBBBB"

Terrakube allow to have one or multiple agents to run jobs, you can have as many agents as you want for a single organization.

To use this feature you could deploy a single executor component using the following values:

# Executor should be enable but we need to customize the apiServiceUrl
executor:
  enabled: true
  apiServiceUrl: "http://terrakube-api-service.terrakube:8080" ## The API is in another namespace called "terrakube"

# We need to disable the default openLdap but we need to provide the internal secret
# so the executor can authenticat with the API and the Registry
security:
  useOpenLDAP: false
  internalSecret: "AxxPdgpCi72f8WhMXCTGhtfMRp6AuBfj"

# We need to disable dex in this deployment
dex:
  enabled: false

# We need to disable default storage MINIO and set some custom values 
# in this example will be deploying like using an external MINIO
#(other backend storage could be used too)
storage:
  defaultStorage: false
  minio:
    accessKey: "admin"
    secretKey: "superadmin"
    bucketName: "terrakube"
    endpoint: "http://terrakube-minio.terrakube:9000" ## MINIO is in another namespace called "terrakube"

# We need to disable API, the default redis and default postgresql database
# But we need to provide some properties like the redis connection
api:
  enabled: false
  defaultRedis: false
  defaultDatabase: false
  properties:
    redisHostname: "terrakube-redis-master.terrakube" ## REDIS is in another namespace called "terrakube"
    redisPassword: "7p9iWVeRV4S944"

# We need to disable registry deployment
registry:
  enabled: false

# We need to disable ui deployment
ui:
  enabled: false

# We need to disable the ingress configuration
# but we need to specify the api and registry URL 
ingress:
  useTls: false
  includeTlsHosts: false
  ui:
    enabled: false
  api:
    enabled: false
    domain: "terrakube-api.minikube.net"
  registry:
    enabled: false
    domain: "terrakube-reg.minikube.net"
  dex:
    enabled: false

Now that we have our values.yaml we can use the following helm command:

helm install --debug --values ./your-values.yaml terrakube terrakube-repo/terrakube -n self-hosted-executor

Now we have a single executor component ready to accept jobs or we could change the number or replicas to have multiple replicas like a pool of agent:

Terrakube components support Open Telemetry by default to enable effective observability.

To enable telemetry inside the Terrakube components please add the following environment variable:

OTEL_JAVAAGENT_ENABLE=true

Once the open telemetry agent is enable we can use other environment variables to setup the monitoring for our application for example to enable jaeger we could add the following using addtional environment variables:

OTEL_TRACES_EXPORTER=jaeger
OTEL_EXPORTER_JAEGER_ENDPOINT=http://jaeger-all-in-one:14250
OTEL_SERVICE_NAME=TERRAKUBE-API

Now we can go the jaeger ui to see if everything is working as expected.

There are several differente configuration options for example:

Jaeger exporter

The Jaeger exporter. This exporter uses gRPC for its communications protocol.

Zipkin exporter

The Zipkin exporter. It sends JSON in Zipkin format to a specified HTTP URL.

Prometheus exporter

The Prometheus exporter.

Open Telemetry Example

One small example to show how to use open telemetry with docker compose can be found in the following URL:

Requirements

• Azure Active Directory here

• Azure Storage Account

For this example lets image that you will be using the following domains to deploy Terrakube.

• registry.terrakube.azure.com

• ui.terrakube.azure.com

• api.terrakube.azure.com

Setup Azure Authentication

You need to complete the Azure authentication setup for Dex. You can found information in this link

You need to go to your Azure and create a new Application

After the application is created you need to add the redirect URL.

You will also need to add the permission Directory.Read.All and ask a Azure administrator to approve the permission.

Now you can create the DEX configuration, you will use this config later when deploying the helm chart.

## Dex
dex:
  config:
    issuer: https://api.terrakube.azure.com/dex
    storage:
      type: memory
    oauth2:
      responseTypes: ["code", "token", "id_token"] 
      skipApprovalScreen: true
    web:
      allowedOrigins: ['*']
  
    staticClients:
    - id: microsoft
      redirectURIs:
      - 'https://ui.terrakube.azure.com'
      - 'http://localhost:10001/login'
      - 'http://localhost:10000/login'
      - '/device/callback'
      name: 'microsoft'
      public: true

    connectors:
    - type: microsoft
      id: microsoft
      name: microsoft
      config:
        clientID: "<<CHANGE_THIS>>"
        clientSecret: "<<CHANGE_THIS>>"
        redirectURI: "https://api.terrakube.azure.com/dex/callback"
        tenant: "<<CHANGE_THIS>>"

The firt step is to clone the repository.

git clone https://github.com/AzBuilder/terrakube-helm-chart.git

Replace <<CHANGE_THIS>> with the real values, create the values.yaml file and run the helm install

## Global Name
name: "terrakube"

## Terrakube Security
security:
  adminGroup: "<<CHANGE_THIS>>" # The value should be a valida azure ad group (example: TERRAKUBE_ADMIN)
  patSecret: "<<CHANGE_THIS>>"  # Sample Key 32 characters z6QHX!y@Nep2QDT!53vgH43^PjRXyC3X 
  internalSecret: "<<CHANGE_THIS>>" # Sample Key 32 characters Kb^8cMerPNZV6hS!9!kcD*KuUPUBa^B3 
  dexClientId: "microsoft"
  dexClientScope: "email openid profile offline_access groups"
  
## Terraform Storage
storage:
  azure:
    storageAccountName: "XXXXXXX" # <<CHANGE_THIS>>
    storageAccountResourceGroup: "XXXXXXX" # <<CHANGE_THIS>>
    storageAccountAccessKey: "XXXXXXX" # <<CHANGE_THIS>>

## Dex
dex:
  config:
    issuer: https://api.terrakube.azure.com/dex
    storage:
      type: memory
    oauth2:
      responseTypes: ["code", "token", "id_token"] 
      skipApprovalScreen: true
    web:
      allowedOrigins: ['*']
  
    staticClients:
    - id: microsoft
      redirectURIs:
      - 'https://ui.terrakube.azure.com'
      - 'http://localhost:10001/login'
      - 'http://localhost:10000/login'
      - '/device/callback'
      name: 'microsoft'
      public: true

    connectors:
    - type: microsoft
      id: microsoft
      name: microsoft
      config:
        clientID: "<<CHANGE_THIS>>"
        clientSecret: "<<CHANGE_THIS>>"
        redirectURI: "https://api.terrakube.azure.com/dex/callback"
        tenant: "<<CHANGE_THIS>>"

## API properties
api:
  enabled: true
  replicaCount: "1"
  serviceType: "ClusterIP"
  properties:
    databaseType: "H2"

## Executor properties
executor:
  enabled: true  
  replicaCount: "1"
  serviceType: "ClusterIP"
  properties:
    toolsRepository: "https://github.com/AzBuilder/terrakube-extensions"
    toolsBranch: "main"

## Registry properties
registry:
  enabled: true
  replicaCount: "1"
  serviceType: "ClusterIP"

## UI Properties
ui:
  enabled: true
  replicaCount: "1"
  serviceType: "ClusterIP"

## Ingress properties
ingress:
  useTls: true
  ui:
    enabled: true
    domain: "ui.terrakube.azure.com"
    path: "/(.*)"
    pathType: "Prefix" 
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      cert-manager.io/cluster-issuer: letsencrypt
  api:
    enabled: true
    domain: "api.terrakube.azure.com"
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
      cert-manager.io/cluster-issuer: letsencrypt
  registry:
    enabled: true
    domain: "registry.terrakube.azure.com"
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
      cert-manager.io/cluster-issuer: letsencrypt
  dex:
    enabled: true
    path: "/dex/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
      cert-manager.io/cluster-issuer: letsencrypt

Run the installation

helm install --debug --values ./values.yaml terrakube ./terrakube-helm-chart/ -n terrakube

You can handle Terrakube users with differen authentication providers like the following

Click the organizations list on the main menu and then click the Create new organization button

Provide organization name and description and click the Create organization button

Then you will redirected to the Organization Settings page where you can define your teams, this is step is important so you can assign the permissions to users, otherwise you won't be able to create workspaces and modules inside the organization.

Requirements

• Google Cloud Identity here

• Gooble Storage Bucket

For this example lets image that you will be using the following domains to deploy Terrakube.

• registry.terrakube.gcp.com

• ui.terrakube.gcp.com

• api.terrakube.gcp.com

Setup Google Authentication

You need to complete the Google authentication setup for Dex. You can found information in this link

You need to go to your GCP projet and create a new OAuth Application you can follow this steps: firts select "APIs & Services => Credentials"

Once inside the "Credentials" page, you will have to create a new OAuth Client

The OAuth application should look like this with the redirect URL "https://api.terrakube.gcp.com/dex/callback"

For Google authentication we need to get the GCP groups so you need to complete this setup.

Include the Domain Wide Delegation inside the admin consol https://admin.google.com/ for the OAuth application

Using the following permission "https://www.googleapis.com/auth/admin.directory.group.readonly"

You can now generate the JSON credentials file for your application, you will use this file later in the helm chart.

Now you can create the DEX configuration, you will use this config later when deploying the helm chart.

## Dex
dex:
  enabled: true
  config:
    issuer: https://api.terrakube.gcp.com/dex #<<CHANGE_THIS>>
    storage:
      type: memory
    oauth2:
      responseTypes: ["code", "token", "id_token"]
    web:
      allowedOrigins: ["*"]

    staticClients:
    - id: google
      redirectURIs:
      - 'https://ui.terrakube.gcp.com' #<<CHANGE_THIS>>
      - 'http://localhost:3000'
      - 'http://localhost:10001/login'
      - 'http://localhost:10000/login'
      - '/device/callback'
      name: 'google'
      public: true

    connectors:
    - type: google
      id: google
      name: google
      config:
        clientID: "<<CHANGE_THIS>>"
        clientSecret: "<<CHANGE_THIS>>"
        redirectURI: "https://api.terrakube.gcp.com/dex/callback"
        serviceAccountFilePath: "/etc/gcp/secret/gcp-credentials" # GCP CREDENTIAL FILE WILL BE IN THIS PATH
        adminEmail: "<<CHANGE_THIS>>"

The firt step is to clone the repository.

git clone https://github.com/AzBuilder/terrakube-helm-chart.git

Replace <<CHANGE_THIS>> with the real values, create the values.yaml file and run the helm install

## Global Name
name: "terrakube"

## Terrakube Security
security:
  adminGroup: "<<CHANGE_THIS>>" # The value should be a gcp group (format: group_name@yourdomain.com example: terrakube_admin@terrakube.org)
  patSecret: "<<CHANGE_THIS>>"  # Sample Key 32 characters z6QHX!y@Nep2QDT!53vgH43^PjRXyC3X 
  internalSecret: "<<CHANGE_THIS>>" # Sample Key 32 characters Kb^8cMerPNZV6hS!9!kcD*KuUPUBa^B3 
  dexClientId: "google"
  dexClientScope: "email openid profile offline_access groups"
  gcpCredentials: |
    ## GCP JSON CREDENTIALS for service account with API Scope https://www.googleapis.com/auth/admin.directory.group.readonly
    {
      "type": "service_account",
      "project_id": "",
      "private_key_id": "",
      "private_key": "",
      "client_email": "",
      "client_id": "",
      "auth_uri": "",
      "token_uri": "",
      "auth_provider_x509_cert_url": "",
      "client_x509_cert_url": ""
    } 


## Terraform Storage
storage:
  gcp:
    projectId: "<<CHANGE_THIS>>"
    bucketName: "<<CHANGE_THIS>>"
    credentials: |
      ## GCP JSON CREDENTIALS for service account with access to write to the storage bucket
      {
        "type": "service_account",
        "project_id": "",
        "private_key_id": "",
        "private_key": "",
        "client_email": "",
        "client_id": "",
        "auth_uri": "",
        "token_uri": "",
        "auth_provider_x509_cert_url": "",
        "client_x509_cert_url": ""
      } 

## Dex
dex:
  enabled: true
  config:
    issuer: https://api.terrakube.gcp.com/dex #<<CHANGE_THIS>>
    storage:
      type: memory
    oauth2:
      responseTypes: ["code", "token", "id_token"]
    web:
      allowedOrigins: ["*"]

    staticClients:
    - id: google
      redirectURIs:
      - 'https://ui.terrakube.gcp.com' #<<CHANGE_THIS>>
      - 'http://localhost:3000'
      - 'http://localhost:10001/login'
      - 'http://localhost:10000/login'
      - '/device/callback'
      name: 'google'
      public: true

    connectors:
    - type: google
      id: google
      name: google
      config:
        clientID: "<<CHANGE_THIS>>"
        clientSecret: "<<CHANGE_THIS>>"
        redirectURI: "https://api.terrakube.gcp.com/dex/callback"
        serviceAccountFilePath: "/etc/gcp/secret/gcp-credentials" # GCP CREDENTIAL FILE WILL BE IN THIS PATH
        adminEmail: "<<CHANGE_THIS>>" 

## API properties
api:
  enabled: true
  replicaCount: "1"
  serviceType: "ClusterIP"
  properties:
    databaseType: "H2"

## Executor properties
executor:
  enabled: true  
  replicaCount: "1"
  serviceType: "ClusterIP"
  properties:
    toolsRepository: "https://github.com/AzBuilder/terrakube-extensions"
    toolsBranch: "main"

## Registry properties
registry:
  enabled: true
  replicaCount: "1"
  serviceType: "ClusterIP"

## UI Properties
ui:
  enabled: true
  replicaCount: "1"
  serviceType: "ClusterIP"

## Ingress properties
ingress:
  useTls: true
  ui:
    enabled: true
    domain: "terrakube-ui.yourdomain.com"
    path: "/(.*)"
    pathType: "Prefix" 
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      cert-manager.io/cluster-issuer: letsencrypt
  api:
    enabled: true
    domain: "terrakube-api.yourdomain.com"
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
      cert-manager.io/cluster-issuer: letsencrypt
  registry:
    enabled: true
    domain: "terrakube-reg.yourdomain.com"
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
      cert-manager.io/cluster-issuer: letsencrypt
  dex:
    enabled: true
    path: "/dex/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
      cert-manager.io/cluster-issuer: letsencrypt

Run the installation

helm install --debug --values ./values.yaml terrakube ./terrakube-helm-chart/ -n terrakube

Global Variables allow you to define and apply variables one time across all workspaces within an organization. For example, you could define a global variable of provider credentials and automatically apply it to all workspaces.

Creating a Global Variable

Once you are in the desired organization, click the Settings button, then in the left menu select the Global Variables option and click the Add global variable button

In the popup, provide the required values. Use the below table as reference:

Finally click the Save global variable button and the variable will be created

You will see the new global variable in the list. And now the variable will be injected in all the workspaces within the organization

Edit a Global Variable

Click the Edit button next to the global variable you want to edit.

Change the fields you need and click the Save global variable button

Delete a Global Variable

Click the Delete button next to the global variable you want to delete, and then click the Yes button to confirm the deletion. Please take in consideration the deletion is irreversible

In Terrakube you can define user permissions inside your organization using teams.

Creating a Team

Once you are in the desired organization, click the Settings button and then in the left menu select the Teams option.

Click the Create team button

In the popup, provide the team name and the permissions assigned to the team. Use the below table as reference:

Finally click the Create team button and the team will be created

Now all the users inside the team will be able to manage the specific resources within the organization based on the permissions you grantted.

Edit a Team

Click the Edit button next to the team you want to edit

Change the permissions you need and click the Save team button

Delete a Team

Click the Delete button next to the team you want to delete, and then click the Yes button to confirm the deletion. Please take in consideration the deletion is irreversible

Organizations are privately shared spaces for teams to collaborate on infrastructure. Basically are logical containers that you can use to define your company hierarchy to organize the workspaces, modules in the private registry and assign fine grained permissions to your teams.

In this section:

Terrakube has two kinds of API tokens: user and team. Each token type has a different access level and creation process, which are explained below.

API tokens are only shown once when you create them, and then they are hidden. You have to make a new token if you lose the old one.

User API Tokens

API tokens may belong directly to a user. User tokens inherit permissions from the user they are associated with.

User API tokens can be generated inside the User Settings.

Click the Generate and API token button

Add some small description to the token and the duration

The new token will be showed and you can copy it to star calling the Terrakube API using Postman or some other tool.

Delete API Tokens

To delete an API token, navigate to the list of tokens, click the Delete button next to the token you wish to remove, and confirm the deletion.

Team API Tokens

API tokens may belong to a specific team. Team API tokens allow access Terrakube, without being tied to any specific user.

To manage the API token for a team, go to Settings > Teams > Edit button on the desired team.

Click the Create a Team Token button in the Team API Tokens section.

Add the token description and duration and click the Generate token button

The new token will be showed and you can copy it to star calling the Terrakube API using Postman or some other tool.

Delete Team API Tokens

To delete an API token, navigate to the list of tokens, click the Delete button next to the token you wish to remove, and confirm the deletion.