1 of 23

Deployment

The easiest way to deploy Terrakube is using our Helm Chart, to learn more about this please review the following:

You can deploy Terrakube using different authentication providers for more information check

Helm Chart

You can deploy Terrakube to any Kubernetes cluster using the helm chart available in the following repository:

https://github.com/AzBuilder/terrakube-helm-chart

Helm Repository

To use the repository do the following:

helm repo add terrakube-repo https://AzBuilder.github.io/terrakube-helm-chart
helm repo update

Minikube

This following will install Terrakube using "HTTP" a few features like the Terraform registry and the Terraform remote state won't be available because they require "HTTPS", to install with HTTPS support locally with minikube check this

Terrakube can be installed in minikube as a sandbox environment to test, to use it please follow this:

Setup Helm Repository

helm repo add terrakube-repo https://AzBuilder.github.io/terrakube-helm-chart
helm repo update

Setup Minikube

minikube start
minikube addons enable ingress
minikube addons enable storage-provisioner
minikube ip

Copy the minikube ip address ( Example: 192.168.59.100)

Setup Terrakube namespace

kubectl create namespace terrakube

Setup DNS records

sudo nano /etc/hosts

192.168.59.100 terrakube-ui.minikube.net
192.168.59.100 terrakube-api.minikube.net
192.168.59.100 terrakube-reg.minikube.net

Install Terrakube

helm install terrakube terrakube-repo/terrakube -n terrakube

It will take some minutes for all the pods to be available, the status can be checked using

kubectl get pods -n terrakube

If you found the following message "Snippet directives are disabled by the Ingress administrator", please update the ingres-nginx-controller configMap in namespace ingress-nginx adding the following:

allow-snippet-annotations: "true"

Reference: https://github.com/AzBuilder/terrakube/issues/618#issuecomment-1838980451

The environment has some users, groups and sample data so you can test it quickly.

Visit http://terrakube-ui.minikube.net and login using admin@example.com with password admin

You can login using the following user and passwords

The sample user and groups information can be updated in the kubernetes secret:

terrakube-openldap-secrets

Minikube will use a very simple OpenLDAP, make sure to change this when using in a real kubernetes environment. Using the option security.useOpenLDAP=false in your helm deployment.

Select the "simple" organization and the "sample_simple" workspace and run a job.

For more advance configuration options to install Terrakube visit:

https://github.com/AzBuilder/terrakube-helm-chart

Minikube + HTTPS

Terrakube can be installed in minikube as a sandbox environment with HTTPS, using terrakube with HTTPS will allow to use the Terraform registry and the Terraform remote state backend to be used locally without any issue.

Please follow these instructions:

Requirements:

Install to generate the local certificates.

We will be using following domains in our test installation:

Generate local CA certificate

Local CA certificate path

Local CA certificate content

Generate certificate for *.minikube.net

These command will generate two files key.pem and cert.pem that we will be using later.

Now we have all the necessary to install Terrakube with HTTPS

Setup Helm Repository

Setup Minikube

Copy the minikube ip address ( Example: 192.168.59.100)

Create Kubernetes secret with local certificate

Setup Terrakube namespace

Setup DNS records

Setup Ingress Certificates

Helm Values.

We need to generate a file called value.yaml with the following using the content of our rootCa.pem from the previous step:

Install Terrakube with local HTTPS support

Using Terrakube

The environment has some users, groups and sample data so you can test it quickly.

Visit https://terrakube-ui.minikube.net and login using admin@example.com with password admin

We should be able to use the UI using a valid certificate.

Connect to Terrakube

Using terraform we can connect to the terrakube api and the private registry using the following command with the same credentials that we used to login to the UI.

Terraform Remote State.

Lets create a simple Terraform file.

Execute Terraform Remotely

Terraform Init:

Terraform apply:

Checking the UI will show:

Ingress Configuration

The default ingress configuration for terrakube can be customize using a terrakube.yaml like the following:

dex:
  config:
    issuer: http://terrakube-api.minikube.net/dex ## CHANGE THIS DOMAIN
    #.....
    #PUT ALL THE REST OF THE DEX CONFIG AND UPDATE THE URL WITH THE NEW DOMAIN
    #.....

## Ingress properties
ingress:
  useTls: false
  ui:
    enabled: true
    domain: "terrakube-ui.minikube.net" ## CHANGE THIS DOMAIN
    path: "/(.*)"
    pathType: "Prefix" 
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
  api:
    enabled: true
    domain: "terrakube-api.minikube.net" ## CHANGE THIS DOMAIN
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
  registry:
    enabled: true
    domain: "terrakube-reg.minikube.net" ## Change to your customer domain
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
  dex:
    enabled: true
    path: "/dex/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"

Update the custom domains and the other ingress configuration depending of your kubernetes environment

The above example is using a simple ngnix ingress

Now you can install terrakube using the command

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

User Authentication (DEX)

To authenticate users Terrakube implement DEX so you can authenticate using differente providers using dex connectors like the following:

Azure Active Direcory
Google Cloud Identity
Amazon Cognito
Github Authentication
Gitlab Authentictaion
OIDC
LDAP
Keycloak
etc.

Any dex connector that implements the "groups" scope should work without any issue

The Terrakube helm chart is using DEX as dependency, so you can quickly implement it using any exiting dex configuration.

Make sure to update the dex configuration when deploying terrakube in a real kubernetes environment, by default it is using a very basic openLDAP with some sample data. To disable udpate security.useOpenLDAP in your terrakube.yaml

To customize the DEX setup just create a simple terrakube.yaml and update the configuration like the following example:

# UPDATE THE DEX CLIENT ID AND SCOPE 
security:
  dexClientId: "example-app"
  dexClientScope: "email openid profile offline_access groups"
  useOpenLDAP: false
  
# UPDATE THE DEX CONFIG
dex:
  config:
    issuer: http://terrakube-api.minikube.net/dex

    storage:
      type: memory
    web:
      http: 0.0.0.0:5556
      allowedOrigins: ['*']
      skipApprovalScreen: true
    oauth2:
      responseTypes: ["code", "token", "id_token"] 

    connectors:
    - type: ldap
      name: OpenLDAP
      id: ldap
      config:
        # The following configurations seem to work with OpenLDAP:
        #
        # 1) Plain LDAP, without TLS:
        host: terrakube-openldap-service:389
        insecureNoSSL: true
        #
        # 2) LDAPS without certificate validation:
        #host: localhost:636
        #insecureNoSSL: false
        #insecureSkipVerify: true
        #
        # 3) LDAPS with certificate validation:
        #host: YOUR-HOSTNAME:636
        #insecureNoSSL: false
        #insecureSkipVerify: false
        #rootCAData: 'CERT'
        # ...where CERT="$( base64 -w 0 your-cert.crt )"

        # This would normally be a read-only user.
        bindDN: cn=admin,dc=example,dc=org
        bindPW: admin

        usernamePrompt: Email Address

        userSearch:
          baseDN: ou=People,dc=example,dc=org
          filter: "(objectClass=person)"
          username: mail
          # "DN" (case sensitive) is a special attribute name. It indicates that
          # this value should be taken from the entity's DN not an attribute on
          # the entity.
          idAttr: DN
          emailAttr: mail
          nameAttr: cn

        groupSearch:
          baseDN: ou=Groups,dc=example,dc=org
          filter: "(objectClass=groupOfNames)"

          userMatchers:
            # A user is a member of a group when their DN matches
            # the value of a "member" attribute on the group entity.
          - userAttr: DN
            groupAttr: member

          # The group name should be the "cn" value.
          nameAttr: cn

    staticClients:
    - id: example-app
      redirectURIs:
      - 'http://terrakube-ui.minikube.net'
      - '/device/callback'
      - 'http://localhost:10000/login'
      - 'http://localhost:10001/login'
      name: 'example-app'
      public: true

Dex configuration examples can be found here.

Storage backend

Azure Storage Account

This guide will assume that you are using the minikube deployment, but the storage backend can be used in any real kubernetes environment.

The first step will be to create one azure storage account with the folling containers:

content
registry
tfoutput
tfstate

Create Azure Storage Account tutorial link

Once the storage account is created you will have to get the "Access Key"

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  azure:
    storageAccountName: "<<STORAGE ACCOUNT NAME>>"
    storageAccountResourceGroup: "<< RESOURCE GROUP >>"
    storageAccountAccessKey: "<< STORAGE ACCOUNT ACCESS KEY"

Now you can install terrakube using the command

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

Amazon Cloud Storage

This guide will assume that you are using the minikube deployment, but the storage backend can be used in any real kubernetes environment.

The first step will be to create one s3 bucket with private access

Create S3 bucket tutorial link

Once the s3 bucket is created you will need to get the following:

access key
secret key
bucket name
region

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  aws:
    accessKey: "rqerqw"
    secretKey: "sadfasfdq"
    bucketName: "qerqw"
    region: "us-east-1"

Now you can install terrakube using the command:

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

Google Cloud Storage

This guide will assume that you are using the minikube deployment, but the storage backend can be used in any real kubernetes environment.

The first step will be to create one google storage bucket with private access

Create storage bucket tutorial link

Once the google storage bucket is created you will need to get the following:

project id
bucket name
JSON GCP credentials file with access to the storage bucket

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  gcp:
    projectId: "sample project"
    bucketName: "sampledata"
    credentials: |
      {
        "type": "service_account",
        "project_id": "XXXXXXX",
        ......
      }

Now you can install terrakube using the command:

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

Minio (S3 compatible)

This guide will assume that you are using the minikube deployment, but the storage backend can be used in any real kubernetes environment.

The first step will be to deploy a minio instance inside minikube in the terrakube namespace

MINIO helm deployment link

Create the file minio-setup.yaml that we can use to create the default user and buckets

auth:
  rootUser: "admin"
  rootPassword: "superadmin"
defaultBuckets: "terrakube"

kubectl install --values minio-setup.yaml miniostorage bitnami/minio -n terrakube

Once the minio storage is installed lets get the service name.

kubectl get svc -o wide -n terrakube

The service name for the minio storage should be "miniostorage"

Once minio is installed with a bucket you will need to get the following:

access key
secret key
bucket name
endpoint (http://miniostorage:9000)

Now you have all the information we will need to create a terrakube.yaml for our terrakube deployment with the following content:

## Terrakube Storage
storage:
  defaultStorage: false
  minio:
    accessKey: "admin"
    secretKey: "superadmin"
    bucketName: "terrakube"
    endpoint: "http://miniostorage:9000"

Now you can install terrakube using the command:

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

Database Backend

WIP

SQL Azure

To use a SQL Azure with your Terrakube deployment create a terrakube.yaml file with the following content:

## Terrakube API properties
api:
  defaultDatabase: false
  loadSampleData: false
  properties:
    databaseType: "SQL_AZURE"
    databaseHostname: "server_name.database.windows.net"
    databaseName: "database_name"
    databaseUser: "database_user"
    databasePassword: "database_password"
    databaseSchema: "dbo"

loadSampleData this will add some organization, workspaces and modules by default in your database if need it

Now you can install terrakube using the command.

helm install --values terrakube.yaml terrakube terrakube-repo/terrakube -n terrakube

PostgreSQL

By default the helmchart deploy a postgresql database in your terrakube namespace but if you want to customize that you can change the default deployment values.

To use a PostgreSQL with your Terrakube deployment create a terrakube.yaml file with the following content:

loadSampleData this will add some organization, workspaces and modules by default in your database if need it

Now you can install terrakube using the command.

Postgresql SSL mode can be use adding databaseSslMode parameter by default the value is "disable", but it accepts the following values; disable, allow, prefer, require, verify-ca, verify-full. This feature is supported from Terrakube 2.15.0. Reference:

MySQL

To use a MySQL with your Terrakube deployment create a terrakube.yaml file with the following content:

loadSampleData this will add some organization, workspaces and modules by default in your database

Now you can install terrakube using the command.

H2

To use a H2 with your Terrakube deployment create a terrakube.yaml file with the following content:INT

H2 database is just for testing, each time the api pod is restarted a new database will be created

loadSampleData this will add some organization, workspaces and modules by default in your database, keep databaseHostname, databaseName, databaseUser and databasePassword empty

Now you can install terrakube using the command.

Custom CA Certs

Adding certificates at runtime

Terrakube componentes (api, registry and executor) are using to create the docker images

When using buildpack to add a custom CA certificate at runtime you need to do the following:

Provide the following environment variable to the container:

Inside the path there is a folder call "ca-certificates"

We need to mount some information to that path

Inside this folder we should put out custom PEM CA certs and one additional file call type

The content of the file type is just the text "ca-certificates"

Finally your helm terrakube.yaml should look something like this because we are mounting out CA certs and the file called type in the following path " /mnt/platform/bindings/ca-certificates"

When mounting the volume with the ca secrets dont forget to add the key "type", the content of the file is already defined inside the helm chart

Checking the terrakube component two additional ca certs are added inside the sytem truststore

Additinal information about buildpacks can be found in this link:

Adding certificate at build time

Terrakube allow to add the certs when building the application, to use this option use the following:

The certs will be added at runtime as the following image.

Custom Terraform CLI Builds

Terrakube will support terraform cli custom builds or custom cli mirrors, the only requirement is to expose an endpoint with the following structure:

Support is available from version 2.12.0

Example Endpoint:

https://eov1ys4sxa1bfy9.m.pipedream.net/

To support custom terrafom cli releases when using the helm chart us the following:

Self-Hosted Agents

This feature is supported from version 2.20.0 and helm chart version 3.16.0

Terrakube allow to have one or multiple agents to run jobs, you can have as many agents as you want for a single organization.

To use this feature you could deploy a single executor component using the following values:

The above values are assuming the we have deploy terrakube using the domain "minikube.net" inside a namespace called "terrakube"

Now that we have our values.yaml we can use the following helm command:

Now we have a single executor component ready to accept jobs or we could change the number or replicas to have multiple replicas like a pool of agent:

Ephemeral Agents

This feature is supported from version 2.22.0

The following will explain how to run the executor component in"ephemeral" mode.

These environment variables can be used to customize the API component:

ExecutorEphemeralNamespace (Default value: "terrakube")
ExecutorEphemeralImage (Defatul value: "azbuilder/executor:2.22.0" )
ExecutorEphemeralSecret (Default value: "terrakube-executor-secrets" )

The above is basically to control where the job will be created and executed and to mount the secrets required by the executor component

Internally the Executor component will use the following to run in "ephemeral" :

EphemeralFlagBatch (Default value: "false")
EphemeralJobData, this contains all the data that the executor need to run.

Requirements

To use Ephemeral executors we need to create the following configuration:

Service Account Creation

apiVersion: v1
kind: ServiceAccount
metadata:
  name: terrakube-api-service-account
  namespace: terrakube

Role Creation

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: terrakube
  name: terrakube-api-role
rules:
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

Role Binding Creation

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: terrakube-api-role-binding
  namespace: terrakube
subjects:
- kind: ServiceAccount
  name: terrakube-api-service-account
  namespace: terrakube
roleRef:
  kind: Role
  name: terrakube-api-role
  apiGroup: rbac.authorization.k8s.io

Helm Chart Configuration

Once the above configuration is created we can deploy the Terrakube API like the following example:

## API properties
api:
  image: "azbuilder/api-server"
  version: "2.22.0"
  serviceAccountName: "terrakube-api-service-account"
  env:
  - name: ExecutorEphemeralNamespace
    value: terrakube
  - name: ExecutorEphemeralImage
    value: azbuilder/executor:2.22.0
  - name: ExecutorEphemeralSecret
    value: terrakube-executor-secrets

Workspace Configuration

Add the environment variable TERRAKUBE_ENABLE_EPHEMERAL_EXECUTOR=1 like the image below

Workspace Execution

Now when the job is running internally Terrakube will create a K8S job and will execute each step of the job in a "ephemeral executor"

Internal Kubernetes Job Example:

Plan Running in a pod:

Apply Running in a different pod:

Node Selector.

If required you can specify the node selector configuration where the pod will be created using something like the following:

api:
  env:
  - name: JAVA_TOOL_OPTIONS
    value: "-Dorg.terrakube.executor.ephemeral.nodeSelector.diskType=ssd -Dorg.terrakube.executor.ephemeral.nodeSelector.nodeType=spot"

The above will be the equivalent to use the Kubernetes YAML like:

  nodeSelector:
    disktype: ssd
    nodeType: spot

Adding node selector configuration is available from version 2.23.0

Proxy Configuration

When Terrrakube needs to run behind a corporate proxy the following environment variable can be used in each container:

JAVA_TOOL_OPTIONS="-Dhttp.proxyHost=your.proxy.net -Dhttp.proxyPort=8080 -Dhttp.nonProxyHosts=XXXXX -Dhttps.proxyHost=your.proxy.net -Dhttps.proxyPort=8080 -Dhttps.nonProxyHosts=XXXXX"

When using the official helm chart the environment variable setting can be set like the following:

api:
  env:
  - name: JAVA_TOOL_OPTIONS
    value: "-Dhttp.proxyHost=your.proxy.net -Dhttp.proxyPort=8080 -Dhttp.nonProxyHosts=XXXXX -Dhttps.proxyHost=your.proxy.net -Dhttps.proxyPort=8080 -Dhttps.nonProxyHosts=XXXXX"
    
registry:
  env:
  - name: JAVA_TOOL_OPTIONS
    value: "-Dhttp.proxyHost=your.proxy.net -Dhttp.proxyPort=8080 -Dhttp.nonProxyHosts=XXXXX -Dhttps.proxyHost=your.proxy.net -Dhttps.proxyPort=8080 -Dhttps.nonProxyHosts=XXXXX"

executor:
  env:
  - name: JAVA_TOOL_OPTIONS
    value: "-Dhttp.proxyHost=your.proxy.net -Dhttp.proxyPort=8080 -Dhttp.nonProxyHosts=XXXXX -Dhttps.proxyHost=your.proxy.net -Dhttps.proxyPort=8080 -Dhttps.nonProxyHosts=XXXXX"

Reference

Token Security

Terrakube use two secrets internally to sign the personal access token and one internal token for intercomponent comunnication when using the helm deployment you can change this values using the following keys:

security:
  patSecret: "AAAAAAAAAAAAAAAAAAAA"
  internalSecret: "BBBBBBBBBBBBBBBBB"

Make sure to change the default values in a real kubernetes deployment.

The secret should be 32 character long and it should be base64 compatible string.

Open Telemetry

Terrakube components support Open Telemetry by default to enable effective observability.

To enable telemetry inside the Terrakube components please add the following environment variable:

OTEL_JAVAAGENT_ENABLE=true

Terrakube API, Registry and Executor support the setup for now from version 2.12.0.

UI support will be added in the future.

Once the open telemetry agent is enable we can use other environment variables to setup the monitoring for our application for example to enable jaeger we could add the following using addtional environment variables:

OTEL_TRACES_EXPORTER=jaeger
OTEL_EXPORTER_JAEGER_ENDPOINT=http://jaeger-all-in-one:14250
OTEL_SERVICE_NAME=TERRAKUBE-API

Now we can go the jaeger ui to see if everything is working as expected.

There are several differente configuration options for example:

Jaeger exporter

The Jaeger exporter. This exporter uses gRPC for its communications protocol.

Zipkin exporter

The Zipkin exporter. It sends JSON in Zipkin format to a specified HTTP URL.

Prometheus exporter

The Prometheus exporter.

For more information please check, the official open telemetry documentation.

Open Telemetry Example

One small example to show how to use open telemetry with docker compose can be found in the following URL:

Minikube + HTTPS

Please follow these instructions:

Requirements:

Install to generate the local certificates.

We will be using following domains in our test installation:

Generate local CA certificate

Local CA certificate path

Local CA certificate content

cat /home/myuser/.local/share/mkcert/rootCA.pem

Generate certificate for *.minikube.net

These command will generate two files key.pem and cert.pem that we will be using later.

mkcert -key-file key.pem -cert-file cert.pem minikube.net *.minikube.net

Created a new certificate valid for the following names 📜
 - "minikube.net"
 - "*.minikube.net"

Reminder: X.509 wildcards only go one level deep, so this won't match a.b.minikube.net ℹ️

The certificate is at "cert.pem" and the key at "key.pem" ✅

It will expire on 19 January 2026 🗓

Now we have all the necessary to install Terrakube with HTTPS

Setup Helm Repository

helm repo add terrakube-repo https://AzBuilder.github.io/terrakube-helm-chart
helm repo update

Setup Minikube

minikube start
minikube addons enable ingress
minikube addons enable storage-provisioner
minikube ip

Copy the minikube ip address ( Example: 192.168.59.100)

Create Kubernetes secret with local certificate

kubectl -n kube-system create secret tls mkcert --key key.pem --cert cert.pem

Setup Terrakube namespace

kubectl create namespace terrakube

Setup DNS records

sudo nano /etc/hosts

192.168.59.100 terrakube-ui.minikube.net
192.168.59.100 terrakube-api.minikube.net
192.168.59.100 terrakube-reg.minikube.net

Setup Ingress Certificates

minikube addons configure ingress

-- Enter custom cert(format is "namespace/secret"): kube-system/mkcert
✅  ingress was successfully configured

minikube addons disable ingress

🌑  "The 'ingress' addon is disabled

minikube addons enable ingress

🔎  Verifying ingress addon...
🌟  The 'ingress' addon is enabled

Helm Values.

We need to generate a file called value.yaml with the following using the content of our rootCa.pem from the previous step:

security:
  caCerts:
    rootCA.pem: |
      -----BEGIN CERTIFICATE-----
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      -----END CERTIFICATE-----

## API properties
api:
  env:
  - name: SERVICE_BINDING_ROOT
    value: /mnt/platform/bindings
  volumes:
    - name: ca-certs
      secret:
        secretName: terrakube-ca-secrets
        items:
        - key: "rootCA.pem"
          path: "rootCA.pem"
        - key: "type"
          path: "type"
  volumeMounts:
  - name: ca-certs
    mountPath: /mnt/platform/bindings/ca-certificates
    readOnly: true
  properties:
    databaseType: "H2"

executor:
  env:
  - name: SERVICE_BINDING_ROOT
    value: /mnt/platform/bindings
  volumes:
    - name: ca-certs
      secret:
        secretName: terrakube-ca-secrets
        items:
        - key: "rootCA.pem"
          path: "rootCA.pem"
        - key: "type"
          path: "type"
  volumeMounts:
  - name: ca-certs
    mountPath: /mnt/platform/bindings/ca-certificates
    readOnly: true

## Registry properties
registry:
  enabled: true
  replicaCount: "1"
  serviceType: "ClusterIP"
  env:
  - name: SERVICE_BINDING_ROOT
    value: /mnt/platform/bindings
  volumes:
    - name: ca-certs
      secret:
        secretName: terrakube-ca-secrets
        items:
        - key: "rootCA.pem"
          path: "rootCA.pem"
        - key: "type"
          path: "type"
  volumeMounts:
  - name: ca-certs
    mountPath: /mnt/platform/bindings/ca-certificates
    readOnly: true

dex:
  config:
    issuer: https://terrakube-api.minikube.net/dex

    storage:
      type: memory
    web:
      http: 0.0.0.0:5556
      allowedOrigins: ['*']
      skipApprovalScreen: true
    oauth2:
      responseTypes: ["code", "token", "id_token"]

    connectors:
    - type: ldap
      name: OpenLDAP
      id: ldap
      config:
        # The following configurations seem to work with OpenLDAP:
        #
        # 1) Plain LDAP, without TLS:
        host: terrakube-openldap-service:1389
        insecureNoSSL: true
        #
        # 2) LDAPS without certificate validation:
        #host: localhost:636
        #insecureNoSSL: false
        #insecureSkipVerify: true
        #
        # 3) LDAPS with certificate validation:
        #host: YOUR-HOSTNAME:636
        #insecureNoSSL: false
        #insecureSkipVerify: false
        #rootCAData: 'CERT'
        # ...where CERT="$( base64 -w 0 your-cert.crt )"

        # This would normally be a read-only user.
        bindDN: cn=admin,dc=example,dc=org
        bindPW: admin

        usernamePrompt: Email Address

        userSearch:
          baseDN: ou=users,dc=example,dc=org
          filter: "(objectClass=person)"
          username: mail
          # "DN" (case sensitive) is a special attribute name. It indicates that
          # this value should be taken from the entity's DN not an attribute on
          # the entity.
          idAttr: DN
          emailAttr: mail
          nameAttr: cn

        groupSearch:
          baseDN: ou=Groups,dc=example,dc=org
          filter: "(objectClass=groupOfNames)"

          userMatchers:
            # A user is a member of a group when their DN matches
            # the value of a "member" attribute on the group entity.
          - userAttr: DN
            groupAttr: member

          # The group name should be the "cn" value.
          nameAttr: cn

    staticClients:
    - id: example-app
      redirectURIs:
      - 'https://terrakube-ui.minikube.net'
      - '/device/callback'
      - 'http://localhost:10000/login'
      - 'http://localhost:10001/login'
      name: 'example-app'
      public: true

## Ingress properties
ingress:
  useTls: true
  includeTlsHosts: true
  ui:
    enabled: true
    domain: "terrakube-ui.minikube.net"
    path: "/"
    pathType: "Prefix"
    tlsSecretName: tls-secret-ui-terrakube
    annotations:
      nginx.ingress.kubernetes.io/use-regex: "true"
  api:
    enabled: true
    domain: "terrakube-api.minikube.net"
    path: "/"
    pathType: "Prefix"
    tlsSecretName: tls-secret-api-terrakube
    annotations:
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
  registry:
    enabled: true
    domain: "terrakube-reg.minikube.net"
    path: "/"
    pathType: "Prefix"
    tlsSecretName: tls-secret-reg-terrakube
    annotations:
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
  dex:
    enabled: true
    path: "/dex/"
    pathType: "Prefix"
    annotations:
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"

Install Terrakube with local HTTPS support

helm install terrakube terrakube-repo/terrakube -n terrakube --values values.yaml

allow-snippet-annotations: "true"

Reference: https://github.com/AzBuilder/terrakube/issues/618#issuecomment-1838980

Using Terrakube

The environment has some users, groups and sample data so you can test it quickly.

Visit https://terrakube-ui.minikube.net and login using admin@example.com with password admin

We should be able to use the UI using a valid certificate.

Connect to Terrakube

Using terraform we can connect to the terrakube api and the private registry using the following command with the same credentials that we used to login to the UI.

terraform login terrakube-api.minikube.net
terraform login terrakube-reg.minikube.net

Terraform Remote State.

Lets create a simple Terraform file.

terraform {
  cloud {
    organization = "simple"
    hostname = "terrakube-api.minikube.net"

    workspaces {
      tags = ["myplayground", "example"]
    }
  }
}

# This resource will destroy (potentially immediately) after null_resource.next
resource "null_resource" "previous" {}

resource "time_sleep" "wait_5_seconds" {
  depends_on = [null_resource.previous]

  create_duration = "5s"
}

# This resource will create (at least) 30 seconds after null_resource.previous
resource "null_resource" "next" {
  depends_on = [time_sleep.wait_5_seconds]
}

Execute Terraform Remotely

Terraform Init:

$ terraform init

Initializing Terraform Cloud...

No workspaces found.
  
  There are no workspaces with the configured tags (example, myplayground)
  in your Terraform Cloud organization. To finish initializing, Terraform needs at
  least one workspace available.
  
  Terraform can create a properly tagged workspace for you now. Please enter a
  name to create a new Terraform Cloud workspace.

  Enter a value: simple


Initializing provider plugins...
- Finding latest version of hashicorp/null...
- Finding latest version of hashicorp/time...
- Installing hashicorp/null v3.2.1...
- Installed hashicorp/null v3.2.1 (signed by HashiCorp)
- Installing hashicorp/time v0.9.1...
- Installed hashicorp/time v0.9.1 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform apply:

$ terraform apply

Running apply in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://terrakube-api.minikube.net/app/simple/simple/runs/40

Waiting for the plan to start...

***************************************
Running Terraform PLAN
***************************************

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # null_resource.next will be created
  + resource "null_resource" "next" {
      + id = (known after apply)
    }

  # null_resource.previous will be created
  + resource "null_resource" "previous" {
      + id = (known after apply)
    }

  # time_sleep.wait_5_seconds will be created
  + resource "time_sleep" "wait_5_seconds" {
      + create_duration = "5s"
      + id              = (known after apply)
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + creation_time = "5s"

Do you want to perform these actions in workspace "simple"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

null_resource.previous: Creating...
null_resource.previous: Creation complete after 0s [id=6182221597368620892]
time_sleep.wait_5_seconds: Creating...
time_sleep.wait_5_seconds: Creation complete after 5s [id=2023-10-19T20:25:22Z]
null_resource.next: Creating...
null_resource.next: Creation complete after 0s [id=9093191930998774410]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.