Set up runners for Windows

Windows Runners allow you to run builds in Pipelines on your own Windows infrastructure, and you won’t be charged for the build minutes used by your self-hosted windows runners.

Prerequisites

Below are the sample chocolatey scripts to install all of the prerequisites required:

1 2 3 4 choco install -y git choco install -y openjdk11 choco install -y dotnetfx --pre choco install git-lfs.install # if you need to use git-lfs features

To improve build times, we recommend installing any other dependencies your Pipelines require in advance, such as nuget, xUnit, nUnit, etc.

Allow unsigned scripts to run in PowerShell

The Windows runner generates PowerShell scripts for cloning the repository and running the script for each step in the pipeline. These scripts are generated when the pipeline is run, preventing them from being digitally signed.

To allow the Windows runners to run unsigned PowerShell scripts, set the PowerShell execution policy of the CurrentUser to either:

  • RemoteSigned (recommended)

  • unrestricted

  • bypass

The RemoteSigned execution policy allows local unsigned (uncertified) scripts to run on the device. This includes any potentially malicious unsigned scripts. Before changing the execution policy, review the execution policies and consider their security implications at Microsoft Docs — PowerShell execution policies.

To check the execution policy for the CurrentUser:

  1. Open Windows PowerShell from the Windows Start menu.

  2. Run the following command, which will return the execution policy for the CurrentUser:

    1 Get-ExecutionPolicy -Scope CurrentUser

To change the execution policy for CurrentUser to RemoteSigned:

  1. In Windows PowerShell, run the following command:

    1 Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
  2. Verify that the change was successful by running Get-ExecutionPolicy and confirm that the CurrentUser has the RemoteSigned execution policy.

    1 Get-ExecutionPolicy -Scope CurrentUser

For information on Microsoft PowerShell execution policies, visit Microsoft Docs — PowerShell: about Execution Policies.

Disable the Windows pagefile and swapfile

Best practice

Before you create a Windows Runner, we strongly recommend disabling swapfile.sys and pagefile.sys in your Windows environment. Having swap enabled can lead to non-deterministic build results in regards to memory and OOMing, meaning that sometimes enough swap is available and a build may pass, while other times not enough swap is available which could make the same build OOM.

Follow the steps below to disable pagefile.sys and swapfile.sys in Windows 10. If the following instructions do not work, consult your distributions documentation to configure your Windows environment:

  1. In Windows, select Start, type Advanced System Settings into the Start menu and press Enter to open it

  2. Select the Advanced tab and then the Settings button in the Performance section of the System Properties dialog.

  3. Select the Advanced tab and then the Change button in the Virtual memory section of the Performance Options dialog.

  4. Unselect Automatically manage paging file size for all drives and select No paging file in the Page file size for each drive section of the Virtual Memory dialog, and then select the Set button.

  5. Select OK and then reboot your system.

Using your runner

Currently we only support running one runner per machine.

  1. Navigate to the Runners page:

    • For Workspace runners, visit Workspace settings > Workspace runners.

    • For Repository runners, visit Repository settings > Runners.

  2. Select Add runner.

  3. From the Runner installation dialog, under System and architecture, select Windows (64bit).

  4. Download the zip file provided in Run step on the Runner installation dialog.

  5. Unzip the zip file to the desired directory, for example: C:\Users\your_user_name\atlassian_runners

  6. Open PowerShell as an administrator, go to the bin directory under your Runner folder, run the command provided in Run step on the Runner installation dialog.

Windows Runners use PowerShell to run pipeline steps on your Windows machine (host device). This allows the runner to execute applications on the host, but does not provide a clean build environment for every step. Any side effects generated by the step (such as, installing any applications, starting a database service, or editing a file outside of the build directory) would potentially affect the next step to be run (including new pipeline runs). To compensate for this, the runner try to empty the build directory empty after each step. It is your responsibility to make sure the scripts you run in each step won’t have a major impact on other steps.


Limitations for Windows Runner

Shared build environment

Runners use a shell to execute the step scripts, and the host machine will be shared by multiple steps that are scheduled to execute on the runner. If a script installs or changes makes a system-wide change to the runner in step, such as installing a new library, then the change will affect all following steps run on the host machine.

Unsupported features

The following features are not supported by self-hosted runners due to limitations on how they are implemented and security complications:

Limitations and workarounds

Cache

  • Pre-defined Docker cache is not supported — Docker and the Pipelines pre-defined Docker cache is not supported for Windows Runners.

  • Share caches between different OS — We recommend specifying different cache name for different runner types, such as a Windows runner and a Linux runner. For example:

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 pipelines: custom: customPipelineWithRunnerAndCache: - step: name: Step 1 runs-on: - 'windows' script: - echo "This step will run on a self hosted windows infrastructure."; caches: - windows_bundler - step: name: Step 2 runs-on: - 'linux' script: - echo "This step will run on a self hosted linux infrastructure."; caches: - linux_bundler - step: name: Step 3 runs-on: - 'linux' script: - echo "This step will run on Atlassian's infrastructure as usual."; caches: - linux_bundler definitions: caches: linux_bundler: vendor/bundle windows_bundler: vendor/bundle

    Caches can contain platform-specific files that do not work on other operating systems. Sharing caches between different operation systems might lead to errors, such as when a Windows runner is trying to use a file that is specifically generated for Linux.

  • Bloated cache folder: Due to performance implications, we do not clean up the cache folder at the end of step execution. This may lead to the size of cache directories increasing rapidly, particularly for a workspace runner. If this occurs, we recommend creating a scheduled task to clean up cache folders on a regular basis. For information on creating scheduled tasks in PowerShell, visit Microsoft Docs — PowerShell New-ScheduledTask cmdlet.

  • Be aware that we don’t restrict where your cache folder is located, so you can store the cache in any directory of the device, including c:\windows. Be mindful about any technical implications of where your cache is defined and make sure your host machine is recoverable.

Test Reporting

There is some additional setup required for .Net test reporting, refer to the following support document for details: Test reporting in Pipelines

Git LFS

In order to use Git LFS, you need to install Git LFS on your hosted machine. If you use chocolatey, tyou can install Git LFS with the following PowerShell commands:

1 2 choco install git-lfs.install git lfs install

Conditional step

The glob path defined in the step condition can only support a forward slash (/) and not a backslash (\) even if the step runs on Windows. So it would look like the example provided below:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 - step: name: step1 runs-on: - self.hosted - windows script: - echo "failing paths" - exit 1 condition: changesets: includePaths: # only xml files directly under path1 directory - "path1/*.xml" # any changes in deeply nested directories under path2 - "path2/**"

SSH Keys

You'll want to set up an SSH key in Bitbucket Pipelines if:

  • your build needs to authenticate with Bitbucket or other hosting services to fetch private dependencies.

  • your deployment needs to authenticate with a remote host or service before uploading artifacts.

  • you want builds to use tools such as SSH, SFTP, or SCP.

For security reasons, a runner will not add your SSH keys to the build environment automatically. If required, SSH keys can be passed to a runner using a secure variable.

There are security risks associated with passing private SSH keys as repository variables:

  • Repository variables are copied to child processes that your pipeline builds may spawn.

  • Secured variables can be retrieved by all users with write access to a repository.

We recommend that you never reuse an SSH key as a repository variable. Generate a new SSH key-pair for Pipelines, so the key can be disabled if it is compromised. It is possible to use deployment variables, which you can use with deployment permissions to control access. For details, see: Variables and secrets — Deployment variables.

To add your SSH key using a secure repository variable with OpenSSH:

  1. Install OpenSSH, such as:

    1 choco install openssh
  2. In PowerShell, generate a new SSH key, such as:

    1 ssh-keygen -t rsa -b 4096 -N '' -f ~/.ssh/my_ssh_key
  3. Encode the private key to base64. Pipelines does not currently support line breaks in environment variables. For example:

    1 [convert]::ToBase64String((Get-Content -path "~/.ssh/my_ssh_key" -Encoding byte))
  4. Add the encoded key as a secure variable. Copy the encoded key from the PowerShell and add it as a secured Bitbucket Pipelines environment variable to the repository:

    1. In the Bitbucket repository, select Repository settings > Repository variables.

    2. Copy the base64-encoded private key from PowerShell.

    3. Paste the encoded key as the value for an environment variable. Make sure to check Secured.

  5. Install the public key on a remote host. You must install the public key on the remote host before Pipelines can authenticate with that host. If you want your Pipelines builds to be able to access other Bitbucket repositories, you need to add the public key to that repository.

    To copy the public key to the remote host using SSH, use the ssh-copy-id  command. This command appends the key to the ~/.ssh/authorized_keys file on the remote host:

    1 ssh-copy-id -i my_ssh_key <username>@<remote_host>

    Where <username> is a user on the remote host.

    To test the SSH access to the server:

    1 ssh -i ~/.ssh/my_ssh_key user@host
  6. Get the host keys and add them to ~/.ssh/known_hosts file in the host virtual machine (VM).
    The known_hosts file contains the DSA host keys of SSH servers accessed by the user. It's important to verify that you're connecting to the correct remote host.

    1. Get the DSA host keys of any remote servers. You can do this by executing the following command:

      1 ssh-keyscan -t rsa server.example.com
    2. Add those keys to the ~/.ssh/known_hosts file in the host VM. You can remove any unrelated lines.

  7. Add and decode the SSH key in the bitbucket-pipelines.yml file. For example:

    1 2 3 4 5 6 7 8 9 pipelines: default: - step: runs-on: - self.hosted - windows script: - [Text.Encoding]::ASCII.GetString([Convert]::FromBase64String($Env:MY_SSH_KEY)) | Out-File -Encoding "ASCII" id_rsa - ssh -T -i ./id_rsa git@bitbucket.org

In the script provided above, we use ./id_rsa instead of ~/.ssh/another_private_key. This ensures that the runner will monitor the file generated in the runner build folder and will attempt to remove it at the end of the step. Any files that are created outside of the runner build folder will not be removed and the runner will leave private keys in ~/.ssh, which will increase the chance of the key being exploited.

There is still a chance that we will not be able to clean up the build folder. We suggest you update the SSH key-pair used in the step on a regular basis to reduce the chance of any of your data being compromised.

Additional Help