Ephemeral Agents

This feature is supported from version 2.22.0

The following will explain how to run the executor component in"ephemeral" mode.

These environment variables can be used to customize the API component:

• ExecutorEphemeralNamespace (Default value: "terrakube")

• ExecutorEphemeralImage (Defatul value: "azbuilder/executor:2.22.0" )

• ExecutorEphemeralSecret (Default value: "terrakube-executor-secrets" )

The above is basically to control where the job will be created and executed and to mount the secrets required by the executor component

Internally the Executor component will use the following to run in "ephemeral" :

• EphemeralFlagBatch (Default value: "false")

• EphemeralJobData, this contains all the data that the executor need to run.

Helm Chat Configuration (Automatic)

To enable the ephemeral executor using the helm chart please use the following options:

api:
  serviceAccountName: "terrakube-api-service-account"
  ephemeralExecution:
    enabled: true

Manual Configuration

To use Ephemeral executors we need to create the following configuration:

Service Account Creation

apiVersion: v1
kind: ServiceAccount
metadata:
  name: terrakube-api-service-account
  namespace: terrakube

Role Creation

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: terrakube
  name: terrakube-api-role
rules:
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

Role Binding Creation

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: terrakube-api-role-binding
  namespace: terrakube
subjects:
- kind: ServiceAccount
  name: terrakube-api-service-account
  namespace: terrakube
roleRef:
  kind: Role
  name: terrakube-api-role
  apiGroup: rbac.authorization.k8s.io

Workspace Configuration

Add the environment variable TERRAKUBE_ENABLE_EPHEMERAL_EXECUTOR=1 like the image below

Workspace Execution

Now when the job is running internally Terrakube will create a K8S job and will execute each step of the job in a "ephemeral executor"

Internal Kubernetes Job Example:

Plan Running in a pod:

Apply Running in a different pod:

Node Selector.

Adding node selector configuration is available from version 2.23.0

If required you can specify the node selector configuration where the pod will be created using something like the following:

api:
  env:
  - name: JAVA_TOOL_OPTIONS
    value: "-Dorg.terrakube.executor.ephemeral.nodeSelector.diskType=ssd -Dorg.terrakube.executor.ephemeral.nodeSelector.nodeType=spot"

The above will be the equivalent to use the Kubernetes YAML like:

  nodeSelector:
    disktype: ssd
    nodeType: spot

Using Environment Variables for Configuration

This feature is supported from version 2.23.0 or 2.24.0

The following environment variables can be used to customize the ephemeral executor adding the following values inside the workspace settings:

• EPHEMERAL_CONFIG_NODE_SELECTOR_TAGS

  • Example: key1=value1;key2=value2

  • Reference

• EPHEMERAL_CONFIG_SERVICE_ACCOUNT

  • Example: myserviceaccount

  • Reference

• EPHEMERAL_CONFIG_ANNOTATIONS

  • Example: key1=value1;key2=value2

  • Reference

• EPHEMERAL_CONFIG_TOLERATIONS

  • Example: key:operator:effect

  • Reference

• EPHEMERAL_CONFIG_MAP_NAME

  • Reference

• EPHEMERAL_CONFIG_MAP_MOUNT_PATH

  • Reference

More information can be found inside this code