General security

Terrakube security is based organizations and groups.

An organization can have one or multiple groups and each group have different kind of access to manage the following options:

• Module

  • Manage terraform modules inside an organization

• VCS

  • Manage private connections to different VCS like Github, Bitbucket, Azure DevOps and Gitlab and handle SSH keys

• Template

  • Manage the custom flows written in Terrakube Configuration Language when running any job inside the platform

• Workspaces

  • Manage the terraform workspaces to run remote terraform operations.

• Providers

  • Manage the terraform providers available inside the platform

Administrator group

There is one special group inside Terrakube called TERRAKUBE_ADMIN, this is the only group that has access to create organizations and grant access to a teams to manage different organization features, you can also customize the group name if you want to use a different name depending on which Dex connector you are using when running Terrakube.

WIP

Special thanks to SolomonHD who has created the Traefik setup example to use Terrakube

Original Reference:

https://gist.github.com/SolomonHD/b55be40146b7a53b8f26fe244f5be52e

To use a PostgreSQL with your Terrakube deployment create a terrakube.yaml file with the following content:

## Terrakube API properties
api:
  defaultDatabase: false
  loadSampleData: false
  properties:
    databaseType: "POSTGRESQL"
    databaseHostname: "server_name.postgres.database.com"
    databaseName: "database_name"
    databaseUser: "database_user"
    databasePassword: "database_password"
    databaseSslMode: "disable"
    databasePort: "5432"

Now you can install terrakube using the command.

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

The easiest way to deploy Terrakube is using our Helm Chart, to learn more about this please review the following:

You can deploy Terrakube using different authentication providers for more information check

The first step will be to create one s3 bucket with private access

Once the s3 bucket is created you will need to get the following:

• access key

• secret key

• bucket name

• region

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

Now you can install terrakube using the command:

You can deploy Terrakube to any Kubernetes cluster using the helm chart available in the following repository:

Helm Repository

To use the repository do the following:

The first step will be to create one google storage bucket with private access

Once the google storage bucket is created you will need to get the following:

• project id

• bucket name

• JSON GCP credentials file with access to the storage bucket

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

Now you can install terrakube using the command:

To use a H2 with your Terrakube deployment create a terrakube.yaml file with the following content:INT

Now you can install terrakube using the command.

The default ingress configuration for terrakube can be customize using a terrakube.yaml like the following:

dex:
  config:
    issuer: http://terrakube-api.minikube.net/dex ## CHANGE THIS DOMAIN
    #.....
    #PUT ALL THE REST OF THE DEX CONFIG AND UPDATE THE URL WITH THE NEW DOMAIN
    #.....

## Ingress properties
ingress:
  useTls: false
  ui:
    enabled: true
    domain: "terrakube-ui.minikube.net" ## CHANGE THIS DOMAIN
    path: "/(.*)"
    pathType: "Prefix" 
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
  api:
    enabled: true
    domain: "terrakube-api.minikube.net" ## CHANGE THIS DOMAIN
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
  registry:
    enabled: true
    domain: "terrakube-reg.minikube.net" ## Change to your customer domain
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
  dex:
    enabled: true
    path: "/dex/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"

Now you can install terrakube using the command

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

The first step will be to deploy a minio instance inside minikube in the terrakube namespace

Create the file minio-setup.yaml that we can use to create the default user and buckets

auth:
  rootUser: "admin"
  rootPassword: "superadmin"
defaultBuckets: "terrakube"

kubectl install --values minio-setup.yaml miniostorage bitnami/minio -n terrakube

Once the minio storage is installed lets get the service name.

kubectl get svc -o wide -n terrakube

The service name for the minio storage should be "miniostorage"

Once minio is installed with a bucket you will need to get the following:

• access key

• secret key

• bucket name

• endpoint (http://miniostorage:9000)

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  minio:
    accessKey: "admin"
    secretKey: "superadmin"
    bucketName: "terrakube"
    endpoint: "http://miniostorage:9000"

Now you can install terrakube using the command:

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

Local DNS entries

Update the /etc/hosts file adding the following entries:

127.0.0.1 terrakube-api
127.0.0.1 terrakube-ui
127.0.0.1 terrakube-executor
127.0.0.1 terrakube-dex
127.0.0.1 terrakube-registry

Running Terrakube Locally.

git clone https://github.com/AzBuilder/terrakube.git
cd terrakube/docker-compose
docker-compose up -d

Terrakube will be available in the following URL:

• http://terrakube-ui:3000

  • Username: [email protected]

  • Password: admin

Docker Compose with Open Telemetry Setup

You can also test Terrakube using the Open Telemetry example setup to see how the components are working internally and check some internal logs

git clone https://github.com/AzBuilder/terrakube.git
cd terrakube/telemetry-compose
docker-compose up -d

This is the high level architecture of the platform

Component descriptions:

• Terrakube API:

  • Expose a JSON:API or GraphQL API providing endpoints to handle:

    • Organizations.

    • Workspace API with the following support:

      • Terraform state history

      • Terraform output history

      • Terraform Variables (public/secrets)

      • Environment Variables (public/secrets)

      • Cron like support to schedule the jobs execution programmatically

    • Job.

      • Custom terraform flows based on Terrakube Configuration Language

    • Modules

    • Providers

    • Teams

    • Teamplate

• Terrakube Jobs:

  • Automatic process that check for any pending job operations in any workspace to trigger a custom logic flow based on Terrakube Configuration Language

• Terrakube Executor:

  • Service that executes the Terrakube job operations written in Terrakube Configuration Language, handle the terraform state and outputs using cloud storage providers like Azure Storage Account

• Terrakube Open Registry:

  • This component allows to expose an open source private repository protected by Dex that you can use to handle your company private terraform modules or providers.

• Cloud Storage:

  • Cloud storage to handle terraform state, outputs and terraform modules used by terraform CLI

• RDBMS:

  • The platform can be used with any database supported by the Liquibase project.

• Security:

  • All authentication and authorization is handle using .

• Terrakube CLI:

  • Go based CLI that can communicate with the Terrakube API and execute operation for organizations, workspaces, jobs, templates, modules or providers

• Terrakube UI:

  • React based frontend to handle all Terrakube Operations.

Terrakube is an open source collaboration platform for running remote infrastructure as code operations using Terraform or OpenTofu that aims to be a complete replacement for close source tools like Terraform Enterprise, Scalr or Env0.

The platform can be easily installed in any kubernetes cluster, you can easily customize the platform using other open source tools already available for terraform (Example: terratag, infracost, terrascan, etc) and you can integrate the tool with different authentication providers like Azure Active Directory, Google Cloud Identity, Github, Gitlab or any other provider supported by .

Terrakube provides different features like the following:

• Terraform module protocol:

  • This allows Terrakube to expose an open source terraform registry that is protected and is private by default using different Dex for the authentication with different providers.

• Terraform provider protocol:

  • This allows Terrakube to expose an open source terraform registry where you can have create your own private terraform providers or create a mirror of the current terraform providers available in the public Hashicorp registry.

• Handle you infrastructure as code using Organization like closed source tools like Terraform Enterprise and Scalr.

• Easily extended by default integrating with many open source tools available today for terraform CLI.

• Support to run the platform using any RDBMS supported by (mysql, postgres, oracle, sql server, etc).

• Plugins support to extends functionality using different cloud providers and open source tools using the Terrakube Configuration Languague (TCL).

• Easily extends the platform using common technologies like react js, java, spring, elide, liquibase, groovy, bash, go.

• Native integration with the most popular VCS like:

  • Github.

  • Azure DevOps

  • Bitbucket

  • Gitlab

• Support for SSH key (RSA and ED25519)

• Handle Personal Access Token

• Extends and create custom flows when running terraform remote operations with custom logics like:

  • Approvals

  • Budget reviews

  • Security reviews

  • Custom logic that you can build with groovy and bash scripts

Organizations are privately shared spaces for teams to collaborate on infrastructure. Basically are logical containers that you can use to define your company hierarchy to organize the workspaces, modules in the private registry and assign fine grained permissions to your teams.

In this section:

The workspace overview page provides you a summary of the current workspace state.

Managing Workspace Tags

You can manage tags for the workspace in the overview page. The right panel lets you search existing tags or type a new tag name to add or remove tags from the workspace.

The docker images are generated using cloud native buildpacks with the maven plugin, more information can be found in these links:

Alpaquita Linux (Experimental)

From Terrakube 2.17.0 we generate 3 additional docker images for the API, Registry and Executor components that are based on that are designed and optimized for Spring Boot applications.

Click the organizations list on the main menu and then click the Create new organization button

Provide organization name and description and click the Create organization button

Then you will redirected to the Organization Settings page where you can define your teams, this is step is important so you can assign the permissions to users, otherwise you won't be able to create workspaces and modules inside the organization.

Terrakube creates the following default templates:

CLI-Driven Templates

Terrakube use job templates for all executions, Terrakube automatically create the following templates that are used when using the terraform remote state backend operations. This templates are created in all organizations.

Terraform Plan/Apply Cli Template

Will be used when you execute terraform apply using the terraform cli

Terraform Plan/Destroy Cli Templates

Will be used when you execute terraform destroyusing the terraform cli

Requirements

The dynamic provider credential setup in Azure can be done with the Terrraform code available in the following link:

Make sure to mount your public and private key to the API container as explained

Validate the following terrakube api endpoints are working:

Set terraform variables using: "variables.auto.tfvars"

Run Terraform apply to create all the federated credential setup in AWS and a sample workspace in terrakube for testing

To test the following terraform code can be used:

Running Example:

When running a job Terrakube will correctly authenticate to Azure without any credentials inside the workspace

Terrakube allows you to customize the UI for each step inside your templates using standard HTML. So you can render any kind of content extracted from your Job execution in the Terrakube UI.

For example you can present the costs using Infracost in a friendly way:

Or present a table with the OPA policies

In order to use UI templates you will need to save the HTML for each template step using the . Terrakube expects the ui templates in the following format.

In the above example the 100 property in the JSON refers to the step number inside your template. In order to save this value from the template you can use the Context extension. For example:

Description

The Resource Details action is designed to show detailed information about a specific resource within the workspace. By using the context of the current state, this action provides a detailed view about the properties and dependencies in a tab within the resource drawer.

Display Criteria

By default this Action is enabled and will appear for all the resources. If you want to display this action only for certain resources, please check .

Usage

• Navigate to the Workspace Overview or the Visual State and click a resource name.

• In the Resource Drawer, you will see a new tab Details with the resource attributes and dependencies.

When Terrrakube needs to run behind a corporate proxy the following environment variable can be used in each container:

When using the official helm chart the environment variable setting can be set like the following:

Description

The Open In Azure Portal action is designed to provide quick access to the Azure portal for a specific resource. By using the context of the current state, this action constructs the appropriate URL and presents it as a clickable button.

Display Criteria

By default, this Action is disabled and when enabled will appear for all the azurerm resources. If you want to display this action only for certain resources, please check .

Usage

• Navigate to the Workspace Overview or the Visual State and click a resource name.

• In the Resource Drawer, click the Open in Azure button.

• You will be able to see the Azure portal for that resource.

If you need to manage your workspaces and execute Jobs from an external CI/CD tool you can use the however a simple approach is to use the existing integrations:

To authenticate users Terrakube implement DEX so you can authenticate using differente providers using dex connectors like the following:

• Azure Active Direcory

• Google Cloud Identity

• Amazon Cognito

• Github Authentication

• Gitlab Authentictaion

• OIDC

• LDAP

• Keycloak

• etc.

The Terrakube helm chart is using DEX as dependency, so you can quickly implement it using any exiting dex configuration.

To customize the DEX setup just create a simple terrakube.yaml and update the configuration like the following example:

# UPDATE THE DEX CLIENT ID AND SCOPE 
security:
  dexClientId: "example-app"
  dexClientScope: "email openid profile offline_access groups"
  useOpenLDAP: false
  
# UPDATE THE DEX CONFIG
dex:
  config:
    issuer: http://terrakube-api.minikube.net/dex

    storage:
      type: memory
    web:
      http: 0.0.0.0:5556
      allowedOrigins: ['*']
      skipApprovalScreen: true
    oauth2:
      responseTypes: ["code", "token", "id_token"] 

    connectors:
    - type: ldap
      name: OpenLDAP
      id: ldap
      config:
        # The following configurations seem to work with OpenLDAP:
        #
        # 1) Plain LDAP, without TLS:
        host: terrakube-openldap-service:389
        insecureNoSSL: true
        #
        # 2) LDAPS without certificate validation:
        #host: localhost:636
        #insecureNoSSL: false
        #insecureSkipVerify: true
        #
        # 3) LDAPS with certificate validation:
        #host: YOUR-HOSTNAME:636
        #insecureNoSSL: false
        #insecureSkipVerify: false
        #rootCAData: 'CERT'
        # ...where CERT="$( base64 -w 0 your-cert.crt )"

        # This would normally be a read-only user.
        bindDN: cn=admin,dc=example,dc=org
        bindPW: admin

        usernamePrompt: Email Address

        userSearch:
          baseDN: ou=People,dc=example,dc=org
          filter: "(objectClass=person)"
          username: mail
          # "DN" (case sensitive) is a special attribute name. It indicates that
          # this value should be taken from the entity's DN not an attribute on
          # the entity.
          idAttr: DN
          emailAttr: mail
          nameAttr: cn

        groupSearch:
          baseDN: ou=Groups,dc=example,dc=org
          filter: "(objectClass=groupOfNames)"

          userMatchers:
            # A user is a member of a group when their DN matches
            # the value of a "member" attribute on the group entity.
          - userAttr: DN
            groupAttr: member

          # The group name should be the "cn" value.
          nameAttr: cn

    staticClients:
    - id: example-app
      redirectURIs:
      - 'http://terrakube-ui.minikube.net'
      - '/device/callback'
      - 'http://localhost:10000/login'
      - 'http://localhost:10001/login'
      name: 'example-app'
      public: true

Templates can become big over time while adding steps to execute additional business logic.

Example before template import:

flow:
- type: "terraformPlan"
  step: 100
  commands:
    - runtime: "GROOVY"
      priority: 100
      before: true
      script: |
        import TerraTag
        new TerraTag().loadTool(
          "$workingDirectory",
          "$bashToolsDirectory",
          "0.1.30")
        "Terratag download completed"
    - runtime: "BASH"
      priority: 200
      before: true
      script: |
        cd $workingDirectory
        terratag -tags="{\"environment_id\": \"development\"}"
- type: "terraformApply"
  step: 300

Terrakube supports having the template logic in an external git repository like the following:

flow:
  - type: "terraformPlan"
    step: 100
    importComands:
      repository: "https://github.com/AzBuilder/terrakube-extensions"
      folder: "templates/terratag"
      branch: "import-template"
  - type: "terraformApply"
    step: 200

Using import templates become simply and we can reuse the logic accross several Terrakube organizations

The sample template can be found here.

Terrakube support sharing the terraform state between workspaces using the following data block.

data "terraform_remote_state" "remote_creation_time" {
  backend = "remote"
  config = {
    organization = "simple"
    hostname = "8080-azbuilder-terrakube-vg8s9w8fhaj.ws-us102.gitpod.io"
    workspaces = {
      name = "simple_tag1"
    }
  }
}

For example it can be used in the following way:

data "terraform_remote_state" "remote_creation_time" {
  backend = "remote"
  config = {
    organization = "simple"
    hostname = "8080-azbuilder-terrakube-vg8s9w8fhaj.ws-us102.gitpod.io"
    workspaces = {
      name = "simple_tag1"
    }
  }
}

resource "null_resource" "previous" {}

resource "time_sleep" "wait_30_seconds" {
  depends_on = [null_resource.previous]

  create_duration = data.terraform_remote_state.remote_creation_time.outputs.creation_time
}

resource "null_resource" "next" {
  depends_on = [time_sleep.wait_30_seconds]
}

The following actions are added by default in Terrakube. However, not all actions are enabled by default. Please refer to the documentation below if you are interested in activating these actions.

General

• Open Documentation: Adds a button to quickly navigate to the Terraform registry documentation for a specific resource.

• Resource Details: Adds a new panel to visualize the state attributes and dependencies for a resource.

Azure

• Open in Azure Portal: Adds a button to quickly navigate to the resource in the Azure Portal.

• Restart Azure VM: Adds a button to restart the Azure VM directly from the workspace overview.

Monitor

• Azure Monitor: Adds a panel to visualize the metrics for the resource.

AI

• Open AI: Adds a button to ask questions or get recommendations using OpenAI based on the workspace data.

Terrakube empowers collaboration between different teams in your organization. To achieve this, you can integrate Terrakube with your version control system (VCS) provider. Although Terrakube can be used with public Git repositories or with private Git repositories using SSH, connecting to your Terraform code through a VCS provider is the preferred method. This allows for a more streamlined and secure workflow, as well as easier management of access control and permissions.

Terrakube supports the following VCS providers:

Webhooks

Terrakube uses webhooks to monitor new commits. This features is not available in SSH and Azure DevOps.

• When new commits are added to a branch, Terrakube workspaces based on that branch will automatically initiate a Terraform job. Terrakube will use the "Plan and apply" template by default, but you can specify a different Template during the Workspace creation.

• When you specify a directory in the Workspace. Terrakube will run the job only if a file changes in that directory

Terrakube can use SSH keys to authenticate to your private repositories. The SSH keys can be uploaded inside the settings menu.

To handle SSH keys inside Terrakube your team should have "Manage VCS settings" access

Terrakube support keys with RSA and ED25519

SSH keys can be used to authenticate to private repositories where you can store modules or workspaces

Once SSH keys are added inside your organization you can use them like the following example:

Modules:

Workspace:

SSH Key Injection

The selected SSH key will be used to clone the workspace information at runtime when running the job inside Terrakube, but it will also be injected to the job execution to be use inside our terraform code.

For example if you are using a module using a GIT connection like the following:

module "test" {
  source = "[email protected]:alfespa17/private-module.git"
}

When running the job, internally terraform will be using the selected SSH key to clone the necesary module dependencies like the below image:

When using a VCS connection you can select wich SSH key should be injected when downloading modules in the workspace settings:

This will allow to use modules using the git format inside a workspace with VCS connection like the following:

module "test" {
  source = "[email protected]:alfespa17/simple-terraform.git"
}

Description

The Azure Monitor Metrics action allows users to fetch and visualize metrics from Azure Monitor for a specified resource.

Display Criteria

By default, this Action is disabled and when enabled will appear for all the azurerm resources. If you want to display this action only for certain resources, please check display criteria.

Setup

This action requires the following variables as Workspace Variables or Global Variables in the Workspace Organization:

• ARM_CLIENT_ID: The Azure Client ID used for authentication.

• ARM_CLIENT_SECRET: The Azure Client Secret used for authentication.

• ARM_TENANT_ID: The Azure Tenant ID associated with your subscription.

• ARM_SUBSCRIPTION_ID: The Azure Subscription ID where the VM is located.

Usage

• Navigate to the Workspace Overview or the Visual State and click on a resource name.

• Click the Monitor tab to view metrics for the selected resource.

• You can view additional details for the metrics using the tooltip.

• You can see more details for each chart by navigating through it.

Description

The Open Documentation action is designed to provide quick access to the Terraform registry documentation for a specific provider and resource type. By using the context of the current state, this action constructs the appropriate URL and presents it as a clickable button.

Display Criteria

By default this Action is enabled and will appear for all the resources. If you want to display this action only for certain resources, please check display criteria.

Usage

• Navigate to the Workspace Overview or the Visual State and click a resource name.

• In the Resource Drawer, Click the Open documentation button

• You will be able to see the documentation for that resource in the Terraform registry

Description

The Restart VM action is designed to restart a specific virtual machine in Azure. By using the context of the current state, this action fetches an Azure access token and issues a restart command to the specified VM. The action ensures that the VM is restarted successfully and provides feedback on the process.

Display Criteria

By default, this Action is disabled and when enabled will appear for all resources that have resource type azurerm_virtual_machine. If you want to display this action only for certain resources, please check display criteria.

Setup

This action requires the following variables as Workspace Variables or Global Variables in the Workspace Organization:

• ARM_CLIENT_ID: The Azure Client ID used for authentication.

• ARM_CLIENT_SECRET: The Azure Client Secret used for authentication.

• ARM_TENANT_ID: The Azure Tenant ID associated with your subscription.

• ARM_SUBSCRIPTION_ID: The Azure Subscription ID where the VM is located.

Usage

• Navigate to the Workspace Overview or the Visual State and click on a resource name.

• In the Resource Drawer, click the "Restart" button.

• The VM will be restarted, and a success or error message will be displayed.

Action Types define where an action will appear in Terrakube. Here is the list of supported types:

Workspace/Action

The action will appear on the Workspace page, next to the Lock button. Actions are not limited to buttons, but this section is particularly suitable for adding various kinds of buttons. Use this type for actions that involves interaction with the Workspace

Workspace/ResourceDrawer/Action

The action will appear for each resource when you click on a resource using the Overview Workspace page or the Visual State Diagram. Use this when an action applies to specific resources, such as restarting a VM or any actions that involve interaction with the resource.

Workspace/ResourceDrawer/Tab

The action will appear for each resource when you click on a resource using the Overview Workspace page or the Visual State Diagram. Use this when you want to display additional data for the resource, such as activity history, costs, metrics, and more.

For this type, theLabel property will be used as the name of the Tab pane.

Description

The Ask Workspace action allows users to interact with OpenAI's GPT-4o model to ask questions and get assistance related to their Terraform Workspace. This action provides a chat interface where users can ask questions about their Workspace's Terraform state and receive helpful responses from the AI.

Display Criteria

By default, this Action is disabled and when enabled will appear for all resources. If you want to display this action only for certain resources, please check display criteria.

Setup

This action requires the following variables as Workspace Variables or Global Variables in the Workspace Organization:

• OPENAI_API_KEY: The API key to authenticate requests to the OpenAI API. Please check this guide in how to get an OPEN AI api key

Usage

• Navigate to the Workspace and click on the Ask Workspace button.

• Enter your questions and click the Send button or press the Enter key.

When working with Terraform at Enterprise level, you need to organize your infrastructure in different collections. Terrakube manages infrastructure collections with workspaces. A workspace contains everything Terraform needs to manage a given collection of infrastructure, so you can easily organize all your resources based in your requirements.

For example, you can create a workspace for dev environment and a different workspace for production. Or you can separate your workspaces based in your resource types, so you can create a workspace for all your SQL Databases and anothers workspace for all your VMS.

You can create unlimited workspaces inside each Terrakube Organization. In this section:

terrakube cli is Terrakube on the command line. It brings organizations, workspaces and other Terrakube concepts to the terminal.

In this section:

Requirements

• A working Keycloak server with a configured realm.

Steps for configuring Terrakube with Keycloak

For configuring Terrakube with Keycloak using Dex the following steps are involved:

• Keycloak client creation and configuration for Terrakube.

• Configure Terrakube so it works with Keycloak.

• Testing the configuration.

Keycloak client creation

Log in to Keycloak with admin credentials and select Configure > Clients > Create. Define the ID as TerrakubeClient and select openid-connect as its protocol. For Root URL use Terrakube's GUI URL. Then click Save.

A form for finishing the Terrakube client configuration will be displayed. These are the fields that must be fulfilled:

• Name: in this example it has the value of TerrakubeClient.

• Client Protocol: it must be set to openid-connect.

• Access Type: set it to confidential.

• Root URL: Terrakube's UI URL.

• Valid Redirect URIs: set it to *

Then click Save.

Notice that, since we set Access Type to confidential, we have an extra tab titled Credentials. Click the Credentials tab and copy the Secret value. It will be used later when we configure the Terrakube's connector.

Depending on your configuration, Terrakube might expect different client scopes, such as openid, profile, email, groups, etc. You can see if they are assigned to TerrakubeClient by clicking on the Client Scopes tab (in TerrakubeClient).

If they are not assigned, you can assign them by selecting the scopes and clicking on the Add selected button.

If some scope does not exist, you must create it before assigning it to the client. You do this by clicking on Client Scopes, then click on the Create button. This will lead you to a form where you can create the new scope. Then, you can assign it to the client.

Terrakube configuration

We have to configure a Dex connector to use with Keycloak. Add a new connector in Dex's configuration, so it looks like this:

This is the simpler configuration that we can use. Let's see some notes about this fields:

• type: must be oidc (OpenID Connect).

• name: this is the string shown in the connectors list in Terrakube GUI.

• issuer: it refers to the Keycloak server. It has the form [http|https]://<KEYCLOAK_SERVER>/auth/realms/<REALM_NAME>

• clientID: refers to the Client ID configured in Keycloak. They must match.

• clientSecret: must be the secret in the Credentials tab in Keycloak..

• redirectURI: has the form [http|https]://<TERRAKUBE_API>/dex/callback. Notice this is the Terrakube API URL and not the UI URL.

• insecureEnableGroups: this is required to enable groups claims. This way groups defined in Keycloak are brought by Terrakube's Dex connector.

{% hint style="info" %} If your users do not have a name set (First Name field in Keycloak), you must tell oidc which attribute to use as the user's name. You can do this giving the userNameKey:

{% endhint %}

Testing Terrakube authentication

When we click on Terrakube's login button we are given the choice to select the connector we want to use:

Click on Log in with TerrakubeClient. You will be redirected to a login form in Keycloak:

After login, you are redirected back to Terrakube and a dialog asking you to grant access is shown

Click on Grant Access. That is the last step. If everything went right, now you should be logged in Terrakube.

When we deploy some infrastructure we can set some other template to execute in some specific time for example to destroy the infrascturture or resize it.

Example

Define the size as global terraform variables for example SIZE_SMALL and SIZE_BIG, this terraform variables will be injected in our future templates "schedule1" and "schedule2" as terraform variables dynamically.

We define a job template that creates the infrasctucture using some small size and create two schedule at the end of the template.

Create Virtual Machine (Example)

We will have 3 templates, one to create the virtual machine and two to resize it

Once the workspace is executed creating the infrasctructure two automatic shedules will be define

Differente jobs will be executed.

Resize Template (schedule1)

Inject the new size using global terraform variables

Resize Template (schedule2)

Inject the new size using global terraform variables

For using repositories from Azure Devops with Terrakube workspaces and modules you will need to follow these steps:

Navigate to the desired organization and click the Settings button, then on the left menu select VCS Providers

Click the Azure Devops button

In the next screen click the link to in Azure Devops

In the Azure Devops page, complete the required fields and click Create application

In the next screen, copy the App ID and Client Secret

Go back to Terrakube to enter the information you copied from the previous step. Then, click the Connect and Continue button.

You will see an Azure Devops window, click the Accept button to complete the connection

Finally, if the connection was established successfully, you will be redirected to the VCS provider’s page in your organization. You should see the connection status with the date and the user that created the connection.

And now, you will be able to use the connection in your workspaces and modules:

Requirements

The dynamic provider credential setup in GCP can be done with the Terrraform code available in the following link:

Make sure to mount your public and private key to the API container as explained

Validate the following terrakube api endpoints are working:

Set terraform variables using: "variables.auto.tfvars"

Run Terraform apply to create all the federated credential setup in GCP and a sample workspace in terrakube for testing

To test the following terraform code can be used:

Running Example

The context object contains data related to the Workspace that you can use inside the action. The properties depends on the .

context.workspace

Contains the workspace information in the Terrakube api format. See the for more details on the properties available.

Examples

• context.workspace.id: Id of the workspace

• context.workspace.attributes.name: Name of the workspace

• context.workspace.attributes.terraformVersion: Terraform version

Available for action types

• workspace/action

• workspace/resourcedrawer/action

• workspace/resourcedrawer/tab

context.settings

Contains the settings that you configured in the display criteria.

Examples

• context.settings.Repository: the value of repository that you set for the setting. Example:

Available for action types

• workspace/action

• workspace/resourcedrawer/action

• workspace/resourcedrawer/tab

context.state

For workspace/action this property contains the full terraform or open tofu state. For workspace/resourcedrawer/action and workspace/resourcedrawer/tab contains only the section of the resource

Available for action types

• workspace/action

• workspace/resourcedrawer/action

• workspace/resourcedrawer/tab

context.apiUrl

Contains the Terrakube api Url. Useful if you want to use the or execute a call to the Terrakube API.

Available for action types

• workspace/action

• workspace/resourcedrawer/action

• workspace/resourcedrawer/tab

For using repositories from GitHub.com with Terrakube workspaces and modules you will need to follow these steps:

Navigate to the desired organization and click the Settings button, then on the left menu select VCS Providers

Click the Github button and then click the Github Cloud option.

In the next screen click the link to register a new OAuth Application in Github

In the Github page, complete the required fields and click Register application

Next, generate a new client secret

Copy the Client Id and Client Secret from Github and go back to Terrakube to complete the required information. Then, click the Connect and Continue button

You will see the Github page, click the Authorize button to complete the connection

Finally, if the connection was established successfully, you will be redirected to the VCS provider’s page in your organization. You should see the connection status with the date and the user that created the connection.

And now, you will be able to use the connection in your workspaces and modules:

For using repositories from Bitbucket.com with Terrakube workspaces and modules you will need to follow these steps:

Navigate to the desired organization and click the Settings button, then on the left menu select VCS Providers

Click the Bitbucket button and then click the Bitbucket Cloud option

Open Bitbucket Cloud and log in as whichever account you want Terrakube to use and click the settings button in your workspace

Click the OAuth consumers menu and then click the Add consumer button

Complete the required fields and click Save button

In the next screen, copy the Key and Secret

Go back to Terrakube to enter the information you copied from the previous step. Then, click the Connect and Continue button.

You will see a Bitbucket window, click the Grant Access button to complete the connection

Finally, if the connection was established successfully, you will be redirected to the VCS provider’s page in your organization. You should see the connection status with the date and the user that created the connection.

And now, you will be able to use the connection in your workspaces and modules:

You can use tags to categorize, sort, and even filter workspaces based on the tags.

You can manage tags for the organization in the Settings panel. This section lets you create and remove tags. Alternatively, you can add existing tags or create new ones in the Workspace overview page.

Creating a Tag

Once you are in the desired organization, click the Settings button, then in the left menu select the Tags option and click the Add global variable button

In the popup, provide the required values. Use the below table as reference:

Finally click the Save tag button and the variable will be created

You will see the new tag in the list. And now you can use the tag in the workspaces within the organization

Edit a Tag

Click the Edit button next to the tag you want to edit.

Change the fields you need and click the Save tag button

Delete a Tag

Click the Delete button next to the tag you want to delete, and then click the Yes button to confirm the deletion. Please take in consideration the deletion is irreversible and the tag will be removed from all the workspaces using it.

Terrakube components support by default to enable effective observability.

To enable telemetry inside the Terrakube components please add the following environment variable:

Once the open telemetry agent is enable we can use other environment variables to setup the monitoring for our application for example to enable jaeger we could add the following using addtional environment variables:

Now we can go the jaeger ui to see if everything is working as expected.

There are several differente configuration options for example:

Jaeger exporter

The exporter. This exporter uses gRPC for its communications protocol.

Zipkin exporter

The exporter. It sends JSON in to a specified HTTP URL.

Prometheus exporter

The exporter.

Open Telemetry Example

One small example to show how to use open telemetry with docker compose can be found in the following URL:

Generate Public and Private Key.

To use Dynamic Provider credentials we need to genera a public and private key that will be use to generate a validate the federated tokens, we can use the following commands

You need to make sure the private key starts with "-----BEGIN PRIVATE KEY-----" if not the following command can be used to transform the private key to the correct format

The public and private key need to be mounted inside the container and the path should be specify in the following environment variables

• DynamicCredentialPublicKeyPath

• DynamicCredentialPrivateKeyPath

Public Endpoints Requirements

To use Dynamic Provider credentials the following public endpoints were added. This endpoint needs to be accessible for your different cloud providers.

Terrakube Environment Variables:

The following environment variables can be used to customize the dynamic credentials configuration:

• DynamicCredentialId = This will be the kid in the JWKS endpoint (Default value: 03446895-220d-47e1-9564-4eeaa3691b42)

• DynamicCredentialTtl= The TTL for the federated token generated internally in Terrakube (Defafult: 30)

• DynamicCredentialPublicKeyPath= The path to the public key to validate the federated tokens

• DynamicCredentialPrivateKeyPath=The path to the private key to generate the federated tokens

Token structure

Terrakube will generate a JWT token internally, this token will be used to authenticate to your cloud provider.

The token structure looks like the following for Azure

The token structure looks like the following for GCP

The token structure looks like the following for AWS

This example show how easy is to integrate Terrakube Jobs in Bitbucket pipelines.

YAML Definition

Add the following snippet to the script section of your bitbucket-pipelines.yml file:

Variables

(*) = required variable.

Examples

Basic example:

Advanced example:

For more information about this pipe please refer to the following .

Terrakube allow to have one or multiple agents to run jobs, you can have as many agents as you want for a single organization.

To use this feature you could deploy a single executor component using the following values:

Now that we have our values.yaml we can use the following helm command:

Now we have a single executor component ready to accept jobs or we could change the number or replicas to have multiple replicas like a pool of agent:

The first step will be to create one azure storage account with the folling containers:

• content

• registry

• tfoutput

• tfstate

Once the storage account is created you will have to get the "Access Key"

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  azure:
    storageAccountName: "<<STORAGE ACCOUNT NAME>>"
    storageAccountResourceGroup: "<< RESOURCE GROUP >>"
    storageAccountAccessKey: "<< STORAGE ACCOUNT ACCESS KEY"

Now you can install terrakube using the command

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

Adding certificates at runtime

Terrakube componentes (api, registry and executor) are using buildpacks to create the docker images

When using buildpack to add a custom CA certificate at runtime you need to do the following:

Provide the following environment variable to the container:

SERVICE_BINDING_ROOT: /mnt/platform/bindings

Inside the path there is a folder call "ca-certificates"

cnb@terrakube-api-678cb68d5b-ns5gt:/mnt/platform/bindings$ ls
ca-certificates

We need to mount some information to that path

/mnt/platform/bindings/ca-certificates

Inside this folder we should put out custom PEM CA certs and one additional file call type

cnb@terrakube-api-678cb68d5b-ns5gt:/mnt/platform/bindings/ca-certificates$ ls
terrakubeDemo1.pem  terrakubeDemo2.pem  type

The content of the file type is just the text "ca-certificates"

cnb@terrakube-api-678cb68d5b-ns5gt:/mnt/platform/bindings/ca-certificates$ cat type
ca-certificates

Finally your helm terrakube.yaml should look something like this because we are mounting out CA certs and the file called type in the following path " /mnt/platform/bindings/ca-certificates"

## Terrakube Security
security:
  caCerts:
    terrakubeDemo1.pem: |
      -----BEGIN CERTIFICATE-----
      MIIDZTCCA.........

      -----END CERTIFICATE-----
    terrakubeDemo2.pem: |
      -----BEGIN CERTIFICATE-----
      MIIDZTCCA.....
      

      -----END CERTIFICATE-----

## API properties
api:
  version: "2.11.2"
  env:
  - name: SERVICE_BINDING_ROOT
    value: /mnt/platform/bindings
  volumes:
    - name: ca-certs
      secret:
        secretName: terrakube-ca-secrets
        items:
        - key: "terrakubeDemo1.pem"
          path: "terrakubeDemo1.pem"
        - key: "terrakubeDemo2.pem"
          path: "terrakubeDemo2.pem"
        - key: "type'
          path: "type"
  volumeMounts:
  - name: ca-certs
    mountPath: /mnt/platform/bindings/ca-certificates
    readOnly: true

Checking the terrakube component two additional ca certs are added inside the sytem truststore

Added 2 additional CA certificate(s) to system truststore
Setting Active Processor Count to 2
Calculating JVM memory based on 5791152K available memory
For more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx5033128K -XX:MaxMetaspaceSize=246023K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 5791152K, Thread Count: 250, Loaded Class Count: 41022, Headroom: 0%)
Enabling Java Native Memory Tracking
Adding 126 container CA certificates to JVM truststore
Spring Cloud Bindings Enabled
Picked up JAVA_TOOL_OPTIONS: -Djava.security.properties=/layers/paketo-buildpacks_bellsoft-liberica/java-security-properties/java-security.properties -XX:+ExitOnOutOfMemoryError -XX:ActiveProcessorCount=2 -XX:MaxDirectMemorySize=10M -Xmx5033128K -XX:MaxMetaspaceSize=246023K -XX:ReservedCodeCacheSize=240M -Xss1M -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics -Dorg.springframework.cloud.bindings.boot.enable=true

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.7.8)

Additinal information about buildpacks can be found in this link:

• https://blog.dahanne.net/2021/02/06/customizing-cloud-native-buildpacks-practical-examples/#Add_certificates_binding_at_runtime:~:text=ca%2Dcertificates-,Add%20certificates%20binding%20at%20runtime,-If%20your%20image

• https://github.com/paketo-buildpacks/ca-certificates

• https://github.com/paketo-buildpacks/spring-boot

Adding certificate at build time

Terrakube allow to add the certs when building the application, to use this option use the following:

git clone https://github.com/AzBuilder/terrakube
cd terrakube
git checkout <<TERRAKUBE-VERSION>>
mv EXAMPLE.pem bindings/ca-certificates

# This script should be run from the root folder
./scripts/build/terrakubeBuild.sh

The certs will be added at runtime as the following image.

To use Terrakube’s VCS features with a self-hosted GitLab Enterprise Edition (EE) or GitLab Community Edition (CE), follow these instructions. For GitLab.com, see the separate instructions.

• Got to the organization settings you want to configure. Select the VCS Providers on the left menu and click the Add VCS provider button on the top right corner.

• Click the Gitlab button and then click the GitLab Enterprise option.

• On the next screen, enter the following information about your GitLab instance:

• Use a different browser tab to access your GitLab instance and sign in with the account that you want Terrakube to use. You should use a service user account for your organization, but you can also use a personal account.

• Navigate to GitLab "User Settings > Applications" page. This page is located at https://<GITLAB INSTANCE HOSTNAME>/profile/applications

• In the GitLab page, complete the required fields and click Save application

• On the next screen, copy the Application ID and Secret

• Go back to Terrakube to enter the information you copied from the previous step. Then, click the Connect and Continue button.

You will be redirected to GitLab. Click the Authorize button to complete the connection.

Finally, if the connection was established successfully, you will be redirected to the VCS provider’s page in your organization. You should see the connection status with the date and the user that created the connection.

And now, you will be able to use the connection in your workspaces and modules:

Terrakube actions allow you to extend the UI in Workspaces, functioning similarly to plugins. These actions enable you to add new functions to your Workspace, transforming it into a hub where you can monitor and manage your infrastructure and gain valuable insights. With Terrakube actions, you can quickly observe your infrastructure and apply necessary actions.

Key Benefits of Terrakube Actions:

• Extend Functionality: Customize your Workspace with additional functions specifics to your needs.

• Monitor Infrastructure: Gain real-time insights into your infrastructure with various metrics and monitoring tools.

• Manage Resources: Directly manage and interact with your resources from within the Workspace.

• Flexibility: Quickly disable an action if it is not useful for your organization or set conditions for actions to appear, such as if a resource is hosted in GCP or if the workspace is Dev.

• Cloud-Agnostic Hub: Manage multi-cloud infrastructure from a single point, providing observability and actionability without needing to navigate individual cloud portals.

Examples of Terrakube Actions:

• Manage Infrastructure: Restart your Azure VM or add an action to quickly navigate to the resource in the Azure portal directly from Terrakube.

• Monitor Metrics: Track and display resource metrics for performance monitoring using tools like Prometheus, Azure Monitor or AWS Cloudwatch.

• Integrate with Tools: Seamlessly integrate with tools like Infracost to display resource and Workspace costs.

• AI Integration: Use AI to diagnose issues in your Workspaces. By integrating with models like OpenAI, Claude, and Gemini, you can quickly resolve errors in your infrastructure.

• Project and Incident Management: Integrate with Jira, GitHub Issues, and ServiceNow to identify Workspaces with pending work items or reported errors.

• Documentation: Link your Confluence documentation or wiki pages in your preferred tool, allowing quick access to documents related to your Workspace infrastructure.

Built-in Actions

Terrakube provides several built-in actions that you can start using some are enabled by default. Please refer to the documentation for detailed information on each action, including usage and setup instructions.

Developing Actions

If you are interested in creating your own actions, you only need knowledge of JavaScript and React, as an action is essentially a React component. For more details please check our quick start guide, or visit our GitHub Terrakube Actions repository to see some examples, contribute new actions or suggest improvements.

Creating an Action

After you develop your action. To create a new action in Terrakube:

1. Navigate to the settings in any Terrakube organization.

1. Click the Add Action button.

1. Provide the required information as detailed in the table below:

1. Click the Save button.

Click Registry in the main menu and then click the Publish module button

Select an existing version control provider or click Connect to a different VCS to configure a new one. See VCS Providers for more details.

Provide the git repository URL and click the Continue button.

In the next screen, configure the required fields and click the Publish Module button.

The module will be published inside the specified organization. On the details page, you can view available versions, read documentation, and copy a usage example.

Releasing New Versions of a Module

To release a new version of a module, create a new release tag to its VCS repository. The registry automatically imports the new version.

Deleting Modules

In the Module details page click the Delete Module button and then click the Yes button to confirm

You can handle Terrakube users with differen authentication providers like the following

Integrate Terrakube with GitHub Actions is easy and you can handle your workspace from GitHub.

The GIT repository will represent a Terrakube Organization and each folder inside the repository will be a new workspace.

There is an example available in the following

Configuration

Add the following snippet to the script section of your github actions file:

Pull Request example

To run a terraform plan using Terrakube templates use the following example:

File: .github/workflows/pull_request.yml

A new workspace will be created for each folder with the file "terrakube.json". For each PR only new folders or folders that has been updated will be evaluated inside Terrakube.

Push Main branch

To run a terraform apply using Terrakube templates use the following example:

File: .github/workflows/push_main.yml

Terrakube Variables

To define terrakube variables to connect to cloud providers it is recommended to use Global Variables inside the organization using the UI. Terraform variables could be define inside a terraform.tfvars inside each folder or you can define inside the Terrakube UI after the workspace creation.

GitHub Action Inputs

(*) = required variable.

Terraform Version Configuration

Create a file called "terrakube.json" and include the terraform version that will be used for the job execution

Build GitHub Action

To build the github action in your local machine use the following.

To use a SQL Azure with your Terrakube deployment create a terrakube.yaml file with the following content:

Now you can install terrakube using the command.

Terrakube will support terraform cli custom builds or custom cli mirrors, the only requirement is to expose an endpoint with the following structure:

Example Endpoint:

• https://eov1ys4sxa1bfy9.m.pipedream.net/

To support custom terrafom cli releases when using the helm chart us the following:

To use a MySQL with your Terrakube deployment create a terrakube.yaml file with the following content:

Now you can install terrakube using the command.

In this section:

Terrakube use two secrets internally to sign the personal access token and one internal token for intercomponent comunnication when using the helm deployment you can change this values using the following keys:

security:
  patSecret: "AAAAAAAAAAAAAAAAAAAA"
  internalSecret: "BBBBBBBBBBBBBBBBB"

In progress

WIP

We use Gitpod to develop the platform. You can have a complete development environment to test all the components and features with one click using the following button.

Running Terrakube in Gitpod is like running the platform in any Kubernetes distribution in Azure, Aws or GCP, so you will be able to use all the features available with one single click without installing any software in your computer and the environment should be ready in only 1 or 2 minute.

Terrakube API uses an in memory database, it will be deleted and preloaded with default information in every startup.

Gitpod Environment Information.

A file called GITPOD.md will have the information for the entire workspace

The configuration is generated dynamically using the following bash script. This is executed by Gitpod in the environment startup process.

./scripts/setupDevelopmentEnvironment.sh

If for some reason we need to execute the script again, it needs to be executed form the /workspace/terrakube folder

The development environment will have the following information:

Start Development Environment

To run all the Terrakube components run the following task:

After a couple of seconds all the components will be running, you can start/stop/restart any component as needed.

Login Development Environment

After the application startup is completed, we can login to the development environment, to get the URL you can execute the following command:

gp url 3000

The Terrakube login will look like this example:

Dex Authentication.

The Gitpod environment is using an OpenLDAP tha is connected to a Dex using the LDAP connector. The OpenLDAP configuration(users and groups) can be found in the following file:

scripts/setup/dex/config-ldap.ldif

The Dex configuration file is generated dynamically and will be available in the following file:

scripts/setup/dex/config-ldap.yaml

The template that is used to generate the Dex configuration file can be found in this path:

scripts/template/dex/template-config-ldap.yaml

Dex login page should look like this example:

Sample Organizations

The development environment will be preloaded with 4 organizations(azure, gcp, aws and simple), this information is loaded inside the API startup and the scripts to load the information can be found in the following xml files

api/src/main/resources/db/changelog/demo-data/aws.xml
api/src/main/resources/db/changelog/demo-data/gcp.xml
api/src/main/resources/db/changelog/demo-data/azure.xml
api/src/main/resources/db/changelog/demo-data/simple.xml

Sample Teams

Depending of the selected organization and user you will see different information available.

Sample Modules

Each organization is preloaded with some modules that can be found in Github like the following:

Templates

Each organization is preloaded with the following templates to run Terrakube jobs:

For more information of how to use templates please refer to the following repository

Workspaces

All the organization have different empty workspaces, the Simple organization can be used for testing because the workspace is using a basic terraform file with the following resources:

• null_resource

• time_sleep

Other workspaces can be created but they will be deleted on each restart unless are added inside the initialization scripts:

api/src/main/resources/db/changelog/demo-data/simple.xml

API Testing

Gitpod has the thunder-client installed to easily test the API without using the UI.

All the environment variables for the collection are generated on each Gitpod workspace startup The environment variables template can be found in the following directory:

scripts/template/thunder-tests/thunderEnvironment.json

To authenticate the thunder client you can use Dex Device Code Authentication using the following request

The response should look like the following and you can use the verification_uri_complete to finish the device authentication.

{
  "device_code": "htlj4w73ftjfplinifqntp7ept6xaratvgauu2nj3nldlcgxjym",
  "user_code": "HLMH-WKXX",
  "verification_uri": "https://5556-azbuilder-terrakube-XXXXX.ws-us54.gitpod.io/dex/device",
  "verification_uri_complete": "https://5556-azbuilder-terrakube-XXXX.ws-us54.gitpod.io/dex/device?user_code=HLMH-WKXX",
  "expires_in": 300,
  "interval": 5
}

Once the Dex device code authentication is completed you can use the following request to get the access token.

Now you can call any method available in the API without the UI

Terraform Login Protocol

Terraform login protocol can be tested using the thunder-client collection:

The response should look similar to the following:

Terraform Module Protocol

There is one example of terraform module protocol inside the thunder-client collection that you can be used for testing purposes executing the following two request.

The response should look like the following:

• Getting versions for the module

• Getting zip file for the module

OpenAPI Spec

The specification can be obtained using the following request:

When creating a Workspace, Terrakube supports 3 workflows types and based on the selected workflow you will need to provide some parameters. Please refer to each workflow section for more reference.

• Version Control workflow: Store your Terraform configuration in a git repository, and trigger runs based on pull requests and merges.

• CLI-driven workflow: Trigger remote Terraform runs from your local command line.

• API-driven workflow: A more advanced option. Integrate Terraform into a larger pipeline using the Terrakube API.

Version Control workflow

Click Workspaces in the main menu and then click the New workspace button

Select the IaC type for now we support terraform and open tofu

Choose the Version control workflow

Select an existing version control provider or click Connect to a different VCS to configure a new one. See VCS Providers for more details.

Provide the git repository URL and click the Continue button.

Configure the workspace settings.

Once you fill the settings click the Create Workspace button.

When using a VCS workflow you can select which branches, folder and action (aka, Default template (VCS push)) the runs will be triggered from when a git push action happens.

The VCS branches accepts a list of branches separated by comma, the first in the list is used as the default for the runs kicked off from UI. The branches are compared as prefixes against the branch included in the payload sent from VCS provider.

You will be redirected to the Workspace page

And if you navigate to the Workspace menu, you will see the workspace in the Workspaces list

Once you create your workspace, Terrakube sets up a webhook with your VCS. This webhook runs a job based on the selected template every time you push new changes to the set workspace branches. However, this feature does not work yet with Azure DevOps VCS provider.

CLI-driven Workflow

Click Workspaces in the main menu and then click the New workspace button

Choose the CLI-driven workflow

Configure the workspace settings.

Once you fill the settings click the Create Workspace button.

You will be redirected to the Workspace page.

And if you navigate to the Workspace menu you will see the workspace in the Workspaces list

API-driven Workflow

Click Workspaces in the main menu and then click the New workspace button

Choose the API-driven workflow

Configure the workspace settings.

Once you fill the settings click the Create Workspace button.

ou will be redirected to the Workspace page.

And if you navigate to the Workspace menu you will see the workspace in the Workspaces list

Terraform provides a public registry, where you can share and reuse your terraform modules or providers, but sometimes you need to define a Terraform module that can be accessed only within your organization.

Terrakube provides a private registry that works similarly to the public Terraform Registry and helps you share Terraform providers and Terraform modules privately across your organization. You can use any of the main VCS providers to maintain your terraform module code.

In this section:

In some special cases it is necesarry to filter the global variables used inside the job execution, if the "inpursEnv" or "inpustTerraform" is not defined all global variables will be imported to the job context.

Template Configuration

flow:
  - type: "customScripts"
    step: 100
    inputsEnv:
      INPUT_IMPORT1: $IMPORT1
      INPUT_IMPORT2: $IMPORT2
    commands:
      - runtime: "BASH"
        priority: 100
        before: true
        script: |
          echo $INPUT_IMPORT1
          echo $INPUT_IMPORT2

$IMPORT1 and $IMPORT2 will be added to the job context as env variable INPUT_IMPORT1 and INPUT_IMPORT2

Global Variables Setup

Running Workspace

Other configuration Options.

Import terraform variables

flow:
  - type: "customScripts"
    step: 100
    inputsTerraform:
      INPUT_IMPORT1: $IMPORT1
      INPUT_IMPORT2: $IMPORT2
    commands:
      ....

Import when importing the template

flow:
  - type: "terraformPlan"
    step: 100
    importComands:
      repository: "https://github.com/AzBuilder/terrakube-extensions"
      folder: "templates/terratag"
      branch: "import-template"
      inputsEnv:
        INPUT_IMPORT1: $IMPORT1
        INPUT_IMPORT2: $IMPORT2
      inputsTerraform:
        INPUT_IMPORT1: $IMPORT1
        INPUT_IMPORT2: $IMPORT2
  - type: "terraformApply"
    step: 200

Import Precedence

workspace env variables > importComands env variables > Flow env variables workspace terraform variables > importComands terraform variables > Flow terraform variables

Requirements

• Azure Active Directory

• Azure Storage Account

For this example lets image that you will be using the following domains to deploy Terrakube.

• registry.terrakube.azure.com

• ui.terrakube.azure.com

• api.terrakube.azure.com

Setup Azure Authentication

You need to complete the Azure authentication setup for Dex. You can found information in this

You need to go to your Azure and create a new Application

After the application is created you need to add the redirect URL.

You will also need to add the permission Directory.Read.All and ask a Azure administrator to approve the permission.

Now you can create the DEX configuration, you will use this config later when deploying the helm chart.

The firt step is to clone the repository.

Replace <<CHANGE_THIS>> with the real values, create the values.yaml file and run the helm install

Run the installation

terrakube cli is Terrakube on the command line. It brings organizations, workspaces and other Terrakube concepts to the terminal.

Installation

You can find installation instructions here

Getting help

When you are working with terrakube cli probably you will need some help with the supported commands and flags and you can use for that, however a better option is to use the same cli to get help using --help flag or -h shortand. The embedded help provides you more detail for each command and some examples you can use as reference

Authentication

Run to authenticate with your account. But first you need to set some environment variables.

Open your terminal and execute the following commands in order to setup your terrakube environment and get authenticated. You can get the server, path, scheme, tenant id and client id values during the

You will receive a code, open the url in a web browser and provide your account details. See below code as reference

Create your Organization

Organizations are privately shared spaces for teams to collaborate on infrastructure.

• You can check the organizations you have access using

• If you don't have an organization created yet, you can create a new one.

The result for the above command should be something like this

For more commands and options in organization see the full documentation for

Working with Teams

Once you create your organization you will probably want to define the teams and permissions for the organization, so you can use the team command for that.

In the previous command we are creating a new team inside our new Organization and for this case we are providing permissions to manage workspaces, modules and providers. In the name flag we are using an Azure AD Group, so all the team members are defined inside the AD group. For more details about teams and permissions you can see .

Create a Workspace

After having given permissions to our teams we can create a workspace.

And define some variables for the created workspace

If you want to avoid entering the organization or the workspace in each command you can use environment variables, so the previous commands would be simplified as follows.

Run a Job

Now that you have a workspace with all the variables defined, you can execute a job, basically a job is a remote terraform apply, plan or destroy. So if you want to run apply we can execute the following command.

After a few minutes the job will finish and you can check the result

Defining your Modules

Usually you will want to define your infrastructure templates as code using terraform and for this you can use the modules so others can reuse them.

Simplifying the commands

In order to simplify the commands when you are working with the cli, you can use shortands and alias and some environment variables. See the following sections for more details.

Using shorthands

Using alias

Using environment variables

Requirements

The dynamic provider credential setup in AWS can be done with the Terrraform code available in the following link:

Make sure to mount your public and private key to the API container as explained

Validate the following terrakube api endpoints are working:

Set terraform variables using: "variables.auto.tfvars"

Run Terraform apply to create all the federated credential setup in AWS and a sample workspace in terrakube for testing

To test the following terraform code can be used:

For using repositories from Gitlab.com with Terrakube workspaces and modules you will need to follow these steps:

Navigate to the desired organization and click the Settings button, then on the left menu select VCS Providers

Click the Gitlab button

In the next screen click the link to in Gitlab

On Gitlab, complete the required fields and click Save application button

In the next screen, copy the Application ID and Secret

Go back to Terrakube to enter the information you copied from the previous step. Then, click the Connect and Continue button.

You will see a Gitlab window, click the Authorize button to complete the connection

Finally, if the connection was established successfully, you will be redirected to the VCS provider’s page in your organization. You should see the connection status with the date and the user that created the connection.

And now, you will be able to use the connection in your workspaces and modules:

Each Terrakube workspace has its own separate state data, used for jobs within that workspace. In Terrakube you can see the Terraform state in the standard provided by the Terraform cli or you can see a , this is a diagram created by Terrakube to represent the resources inside the JSON state.

JSON State

Go to the workspace and click the States tab and then click in the specific state you want to see.

You will see the state in JSON format

You can click the Download button to get a copy of the JSON state file

Visual State

Inside the states page, click the diagram tab and you will see the visual state for your terraform state.

The visual state will also show some information if you click each element like the following:

Resources

All the resources can be shown in the overview page inside the workspace:

If you click the resources you can check the information about each resouce like the folloing