Reducing server resource usage on ephemeral agent startup

Summary

When an ephemeral agent starts up, it fetches the files it needs to perform a build or deployment from the Bamboo server and saves them in the following subfolders inside its own home directory:

• classpath

• framework-bundles

• plugins

Fetching these files on every ephemeral agent startup may put your Bamboo server under additional load and increase the ephemeral agent's overall startup time. You can decrease the load on the server and improve ephemeral agent startup times by caching the required files on a Kubernetes PersistentVolume (PV) and configuring your ephemeral agent templates to access the volume through a PersistentVolumeClaim (PVC). Learn more about persistent volumes in Kubernetes

Alternatively, set up a reverse proxy with agent dependency caching. Learn how to enable agent dependency caching

Solution

To create a PersistentVolume (PV) and PersistentVolumeClaim (PVC) and configure an ephemeral agent to use the PVC:

1. Start an SSH session with an existing remote agent and copy the classpath, framework-bundles, and plugins directories and their content from the agent's home directory to your local machine.

2. In an easily accessible location on your local machine, configure the PV by creating a new pv.yaml file and paste the following content into it:

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 apiVersion: v1 kind: PersistentVolume metadata: name: my-pv spec: capacity: storage: 1Gi accessModes: - ReadOnlyMany hostPath: path: <PATH_ON_HOST> # Replace with the path to the agent's files on your local machine claimRef: namespace: <NAMESPACE> # Replace with the namespace of the PVC name: <PVC_NAME> # Replace with the name of the PVC

   Replace:

   • <PATH_ON_HOST> with the absolute path to the location on your local machine where you've saved the previously copied files

   • <NAMESPACE> with the PVC namespace

   • <PVC_NAME> with the name of the PVC

   The claimRef field establishes a direct relationship between the PersistentVolume and PersistentVolumeClaim, ensuring that a PersistentVolumeClaim is bound to a specific PersistentVolume.

   To store the agent's files on an external NFS server, upload the agent's files to the server and replace the entire hostPath field with an nfs field as follows:

   nfs: server: <REMOTE_SERVER_IP> # Replace with the IP or hostname of the remote server path: <PATH_ON_REMOTE_SERVER> # Replace with the path to the storage location on the remote server

   Replace:

   • <REMOTE_SERVER_IP> with the IP or hostname of the remote NFS server

   • <PATH_ON_REMOTE_SERVER> with the absolute path to the location where you've saved the previously copied files

3. Apply the PV configuration to your Kubernetes cluster by running the following command:

   1 kubectl apply -f pv.yaml

4. In the same location where you created the pv.yaml file, configure the PVC by creating a pvc.yaml file and pasting the following content into it:

   1 2 3 4 5 6 7 8 9 10 apiVersion: v1 kind: PersistentVolumeClaim metadata: name: `<PVC_NAME>` spec: accessModes: - ReadOnlyMany resources: requests: storage: 1Gi

   Replace <PVC_NAME> with the name of the PVC that you used previously.

5. Apply the PVC configuration to your Kubernetes cluster by running the following command:

   1 kubectl apply -f pvc.yaml

6. In Bamboo, create a new ephemeral agent template or modify an existing one and make it use the PVC you created earlier.

   • Learn more about ephemeral agent templates

   • Learn how to create an ephemeral agent template

   • Learn how to modify an existing template

   In the template's YAML configuration:

   1. Under .spec.volumes, define a new volume with a unique name property value (for example, agent-cache).

   2. Associate the volume with the PVC you created earlier by declaring the persistentVolumeClaim property with a claimName equal to the name specified in the PVC configuration file.

   3. Under .spec.containers.<EPHEMERAL_AGENT_CONTAINER>, add a volumes field and give it a name and mountPath.

   4. Under .spec.containers.<EPHEMERAL_AGENT_CONTAINER>.env, declare a new environment variable with a name property equal to BAMBOO_AGENT_CLASSPATH_DIR and value property equal to the mount path you added in the previous step.

      Here's an example ephemeral agent template configuration:

      1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 --- apiVersion: v1 kind: Pod metadata: name: '{{NAME}}' labels: '{{RESOURCE_LABEL}}': <VALUE> spec: volumes: - name: agent-cache persistentVolumeClaim: claimName: my-pvc containers: - image: atlassian/bamboo-agent-base:<YOUR_BAMBOO_VERSION> name: '{{BAMBOO_AGENT_CONTAINER_NAME}}' env: - name: BAMBOO_EPHEMERAL_AGENT_DATA value: '{{BAMBOO_EPHEMERAL_AGENT_DATA_VAL}}' - name: BAMBOO_AGENT_CLASSPATH_DIR value: <AGENT_CACHE_MOUNTH_PATH> volumeMounts: - name: agent-cache mountPath: <AGENT_CACHE_MOUNTH_PATH> restartPolicy: Never

7. Save the configuration.

   You can now use the ephemeral agent template you just created or modified to run a build or deployment.

   You can verify whether your cache was used by checking the pod log. There should be entries similar to the following:

   1 2 3 4 5 6 7 8 9 10 11 12 13 YYYY-MM-DD HH:MM:SS + '[' -d /agentCache ']' YYYY-MM-DD HH:MM:SS + for SUBDIR in classpath plugins framework-bundles YYYY-MM-DD HH:MM:SS + cp -R /agentCache/classpath /var/atlassian/application-data/bamboo-agent/classpath YYYY-MM-DD HH:MM:SS + chown -R bamboo /var/atlassian/application-data/bamboo-agent/classpath YYYY-MM-DD HH:MM:SS + chmod -R u+w /var/atlassian/application-data/bamboo-agent/classpath YYYY-MM-DD HH:MM:SS + for SUBDIR in classpath plugins framework-bundles YYYY-MM-DD HH:MM:SS + cp -R /agentCache/plugins /var/atlassian/application-data/bamboo-agent/plugins YYYY-MM-DD HH:MM:SS + chown -R bamboo /var/atlassian/application-data/bamboo-agent/plugins YYYY-MM-DD HH:MM:SS + chmod -R u+w /var/atlassian/application-data/bamboo-agent/plugins YYYY-MM-DD HH:MM:SS + for SUBDIR in classpath plugins framework-bundles YYYY-MM-DD HH:MM:SS + cp -R /agentCache/framework-bundles /var/atlassian/application-data/bamboo-agent/framework-bundles YYYY-MM-DD HH:MM:SS + chown -R bamboo /var/atlassian/application-data/bamboo-agent/framework-bundles YYYY-MM-DD HH:MM:SS + chmod -R u+w /var/atlassian/application-data/bamboo-agent/framework-bundles

   Learn how to check the log for a pod