Set up runners for MacOS

MacOS Runners allow you to run builds in Pipelines on your own MacOS infrastructure, and you won’t be charged for the build minutes used by your self-hosted runners.

Prerequisites

  • OpenJDK 11 (11.0.11 or newer) is installed

  • Git is installed

  • A 64-Bit MacOS instance with at least 8GB of RAM as a host for the runner

Using your runner

Currently we only support running one runner per machine.

  1. Download the tar.gz file provided in the Run step on the Runner installation dialog.

  2. Untar the file to the desired directory, for example: /Users/your_user_name/atlassian_runner

  3. Open Terminal (or shell of your choice), go to the bin directory under your Runner folder, run the command provided in Run step on the Runner installation dialog.

MacOS Runners is using the shell and will run directly in your instance which means it is not a clean build environment for every step, and any side effects generated by the step (such as, installing any applications, starting a db service, or editing a file outside of the build directory) would potentially affect the next step to be run. Because of this, the runner will only make a best effort to keep the build directory empty. It is your responsibility to make sure the script you run in your step won’t have a major impact on other steps.


Limitations for MacOS Runner

Memory used by the Runner

As the runner is non-containerized, processes created by steps that are run on the runner will likely use swap memory. As it is not possible to completely disable swap memory, it could produce non-deterministic build results in regards to memory and OOMing, meaning that sometimes enough swap is available and a build may pass, while other times not enough swap is available which could make the same build OOM. Be sure to check for any memory issues due to this limitation.

Shared build environment

The MacOS Runner uses a shell to execute the step scripts, and the host machine will be shared by multiple steps that are scheduled to execute on the runner. If you install/change something globally in your step, such as installing a new library, then this change will affect the next step run on the host machine.

Unsupported features

The following are the features that won't be support by self-hosted MacOS Runners due to limitations on how MacOS Runners is implemented and security complications:

  • SSH keys: For more information, refer to Variables and secrets

  • Pipes: For more information, refer to What are pipes?

  • Service containers: For more information, refer to Databases and service containers

  • Custom build images on AWS ECR

  • Custom Docker in Docker image

  • Step size: Given that MacOS Runners runs directly on your MacOS instance, we don’t put any restriction on memory usage.

  • Overriding the default user

Other limitations and workarounds

Cache

  • Pre-defined Docker cache is not supported: Given that Docker is not supported for MacOS Runners, pre-defined Docker cache is also not supported in MacOS Runners.

  • Share caches between different OS: As a best practice, we recommend specifying a different cache name for each different Runner type, such as a Linux runner and a MacOS runner, because caches could contain platform-specific files that do not work in other operating systems. Sharing caches between different operation systems might lead to errors occurring when a MacOS runner is trying to use a .dll file that is specifically generated for Linux.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 pipelines: custom: customPipelineWithRunnerAndCache: - step: name: Step 1 runs-on: - 'macos' script: - echo "This step will run on a self hosted runner."; caches: - macos_bundler - step: name: Step 2 script: - echo "This step will run on Atlassian's infrastructure as usual."; caches: - linux_bundler definitions: caches: linux_bundler: vendor/bundle macos_bundler: vendor/bundle
  • Bloated cache folder: Due to performance implications, we do not clean up the cache folder at the end of step execution which means the size of related cache directories could increase rapidly, especially for a workspace runner. If that occurs, we recommend creating a scheduled task to clean up cache folders on a regular basis. You can refer to the following documentation created by Apple to see how to setup a scheduled task with Launchd: Scheduling Timed Jobs.

  • Be aware that we don’t put any restrictions on where your cache folder is located, so technically you can define whatever folder you like. Be mindful about any technical implications of where your cache is defined and make sure your hosted machine is recoverable.

Git LFS

In order to use Git LFS, you need to make sure you have Git LFS installed on your hosted machine. If you use Homebrew, then you can use the following script to setup Git LFS:

1 brew install git-lfs

SSH Keys

You'll want to set up an SSH key in Bitbucket Pipelines if:

  • your build needs to authenticate with Bitbucket or other hosting services to fetch private dependencies.

  • your deployment needs to authenticate with a remote host or service before uploading artifacts.

  • you want builds to use tools such as SSH, SFTP, or SCP.

However, due to security reasons, a MacOS Runner will not add your SSH keys to the build environment automatically like a cloud runner does. But you are still able to do it manually if you want to, below are the example steps to add your SSH key manually with OpenSSH:

  • Generate an SSH key (if necessary) 
    You can run the following in a terminal to generate an RSA key pair:

    1 $ ssh-keygen -t rsa -b 4096 -N '' -f my_ssh_key
  • Encode a private key with base64
    Pipelines does not currently support line breaks in environment variables, so you can encode the private key with base64 by running the following:

    1 [convert]::ToBase64String((Get-Content -path "my_ssh_key" -Encoding byte))
  • Add the key as a secure variable
    Copy the encoded key from the terminal and add it as a secured Bitbucket Pipelines environment variable to the repository:

    1. In the Bitbucket repository, select Repository settings > Repository variables.

    2. Copy the base64-encoded private key from the terminal.

    3. Paste the encoded key as the value for an environment variable. Make sure to check Secured.

There are security risks associated with passing private SSH keys as repository variables:

  • Repository variables get copied to child processes that your pipelines build may spawn.

  • Secured variables can be retrieved by all users with write access to a repository.

We recommend that you never pass your own personal SSH key as a repository variable, but instead, generate a new SSH key-pair for Pipelines that can easily be disabled if it is compromised. It may also be worth using deployment variables, which you can combine with deployment permissions to control access. Learn more about deployment variables.

  • Install the public key on a remote host
    You must install the public key on the remote host before Pipelines can authenticate with that host. If you want your Pipelines builds to be able to access other Bitbucket repositories, you need to add the public key to that repository.

    Remote hosts
    If you have SSH access to the server, you can use the ssh-copy-id  command. Typically, the command appends the key to the ~/.ssh/authorized_keys file on the remote host:

    1 ssh-copy-id -i my_ssh_key username@remote_host

    Test the SSH access to the server:

    1 ssh -i ~/.ssh/my_ssh_key user@host
  • Get the host keys and add them to ~/.ssh/known_hosts file in the host virtual machine (VM).
    The known_hosts file contains the DSA host keys of SSH servers accessed by the user. It's important to verify that you're connecting to the correct remote host.

    1. Get the DSA host keys of any remote servers. You can do this by executing the following command:

      1 ssh-keyscan -t rsa server.example.com
    2. Add those keys to the ~/.ssh/known_hosts file in the host VM. You can remove any unrelated lines.

  • Tie everything together in the bitbucket-pipelines.yml file (as shown in the example below):

    1 2 3 4 5 6 7 image: node:6 # specify your Docker image here pipelines: default: - step: script: - (umask 077 ; echo $MY_SSH_KEY | base64 --decode > ./id_rsa) - ssh -i ./id_rsa <user>@<host> 'echo "connected to `host` as $USER"'

In the script provided above, we use ./id_rsa instead of ~/.ssh/another_private_key. This ensures that the MacOS Runner will monitor the file that gets generated in the runner build folder and will do its best to clean it up at the end of the step. Any files that are created outside of the runner build folder will not be removed and will leave private keys in ~/.ssh which will increase the chance of the key getting exploited.

There is still a chance that we will not able to clean up the build folder. We suggest you update the SSH key-pair used in the step on a regular basis to reduce the chance of any of your data being compromised.

Additional Help