Scheduling Code Stream Pipeline Executions using Kubernetes CronJobs

In this blog post I’m going to show how I used a Kubernetes CronJob and the Code Stream CLI to schedule the execution of a Code Stream Pipeline. We use Code Stream to rebuild our demo environments every morning, the pipelines are currently executed by a scheduled script. Being able to use a CronJob means we can manage the configuration as code and automate the execution with job history.

Getting an API Token

It’s always best practice to use the minimum required priviliges, so I’ll generate an API token scoped to “Code Stream Executor” role (if you want to list pipelines and executions using the CLI you’ll also need the Viewer, or User role):

Using the Code Stream CLI

The first step is to set up the CLI, I’m going to use the Docker image (install docs for Code Stream CLI). To do this I’ll download the latest version of the CLI container image, and set up some environment variables for authentication. (For on-premesis authentication you’ll need to use different variables, see the configuration documentation for Code Stream CLI)

docker pull sammcgeown/cs-cli:latest
export CS_SERVER="api.mgmt.cloud.vmware.com"
export CS_APITOKEN="HKWuub2QkgJt...snip...KnxhySSVaMJEm1UWFddW"

Now we can use the CLI to look for the Pipeline ID, and grab the input form to execute it:

docker run -e CS_SERVER="$CS_SERVER" -e CS_APITOKEN="$CS_APITOKEN" sammcgeown/cs-cli:latest get pipeline --name="Test" --ignoreCertificateWarnings
+--------------------------------------+------+-----------+-------------+
|                  ID                  | NAME |  PROJECT  | DESCRIPTION |
+--------------------------------------+------+-----------+-------------+
| 97200467-529a-4bc8-b34d-bddb83c87e3b | Test | CMAC Labs |             |
+--------------------------------------+------+-----------+-------------+
docker run -e CS_SERVER="$CS_SERVER" -e CS_APITOKEN="$CS_APITOKEN" sammcgeown/cs-cli:latest get pipeline --name="Test" --form --ignoreCertificateWarnings
{
  "input1": "",
  "input2": "",
  "input3": ""
}

The input form will be different for each pipeline, depending on the inputs. The JSON format outputted by the CLI needs to be flattened to a single string and the quotes escaped so that it can be used in the command line later – e.g. the output above becomes {\"input1\": \"value\", \"input2\": \"value\", \"input3\": \"value\"}

Create Kubernetes components

Create a namespace in which to run the CronJobs, and a secret to contain the environment variables used by the container:

kubectl create namespace scheduled-tasks
kubectl create secret generic codestream-credentials --from-literal=CS_SERVER=api.mgmt.cloud.vmware.com --from-literal=CS_APITOKEN="HKWuub2QkgJt...snip...KnxhySSVaMJEm1UWFddW"

Next I create the CronJob itself, which has a schedule (when to run the task) and a jobTemplate spec (which defines the task to run).

• schedule conforms to standard cron formatting, this example will execute the pipeline every 5 minutes
• concurrencyPolicy is set to Forbid which means the jobs won’t run concurrently
• template.spec.containers definition will execute the CLI container image
  • command is the command to execute on the container using the container ID and inputs JSON
  • envFrom creates environment variables from the secret created earlier

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: cs-schedule
  namespace: scheduled-tasks
spec:
  schedule: "*/5 * * * *"
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: cs-cli
            image: sammcgeown/cs-cli:latest
            imagePullPolicy: IfNotPresent
            command:
            - "/cs-cli"
            - "--ignoreCertificateWarnings"
            - "create"
            - "execution"
            - "--id"
            - "97200467-529a-4bc8-b34d-bddb83c87e3b"
            - "--inputs"
            - "{\"input1\": \"value\", \"input2\": \"value\", \"input3\": \"value\"}"
            - "--comments"
            - "\"Execution by CronJob cs-schedule on Cluster sc2vc03-tmm-services\""
            envFrom:
            - secretRef:
                name: codestream-credentials
          restartPolicy: OnFailure

The CronJob YAML is applied using kubectl apply -f cs-schedule.yaml. We can view the cronjobs.batch and jobs.batch to check on the execution, and view the logs of the pod (remember you might need to wait until the 5-minute mark, or whatever your schedule is!)

kubectl get all -n scheduled-tasks
NAME                               READY   STATUS      RESTARTS   AGE
pod/cs-schedule-1643122800-6dv2c   0/1     Completed   0          7m53s
pod/cs-schedule-1643123100-j6nqc   0/1     Completed   0          2m52s

NAME                               COMPLETIONS   DURATION   AGE
job.batch/cs-schedule-1643122800   1/1           3s         3m57s

NAME                        SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cronjob.batch/cs-schedule   */5 * * * *   False     0        4m2s            8m46s

kubectl logs cs-schedule-1643123100-j6nqc -n scheduled-tasks
time="2022-01-25 15:05:11" level=info msg="Execution /codestream/api/executions/14063405-092b-43d4-9239-d9b4e10918a8 created"

I can see from the logs output that the execution was started successfully – and if I look in the Executions page of the Code Stream console, I can see the executions running on schedule: