A runner is in an unhealthy state and cannot run pipelines

Summary

A runner is in an unhealthy state and cannot run pipelines

Diagnosis

The status of a runner is Unhealthy and the Runners page is showing the following message:

Cause

At the end of each step in a pipeline, the runner cleans up build directories (folders) before running the next step. If the runner can’t perform this cleanup, it will enter an Unhealthy state. Some possible causes of this issue are:

• The permissions for one or more files (or directories) within a build directory are preventing the runner from deleting files.

• A program or service is accessing one or more files (or directories) in the build directory at the time of cleanup.

Solution

To find out which file or directory is preventing the runner from performing a build cleanup, review the runners' log files. Search for an error, such as:

A detailed error log that can be used for debugging will be displayed below this error message.

Accessing the log files

The location of the runner log files varies between operating systems and the type of runner.

Accessing the log files for Linux Docker or MacOS Shell runners

1. Access the Linux device hosting the affected runner.

2. For MacOS only: Using Terminal or Finder, navigate to the runner installation directory (atlassian-bitbucket-pipelines-runner). This directory will be stored where Terminal was working when the runner was started. For example: if Terminal was working in the users' home directory (~/), such as: /Users/Username/; the runner installation directory would be /Users/Username/atlassian-bitbucket-pipelines-runner/

3. Open the runner.log file using a text editor. This log file is stored in the runners' working directory, in a subdirectory named using the runners' UUID (a 32-character hexadecimal string). Such as:

   1 <runner_working_directory>/<runnerUuid>/runner.log

   For example:

   1 /tmp/b609961f-891e-c872-c36b-f3f2c315d186/runner.log

   The default working directory for the runner is the /tmp directory; with MacOS the /temp subdirectory of the atlassian-bitbucket-pipelines-runner directory.

4. Review the end of the log file for errors that need to be addressed.

Fixing the permissions of a file or folder on Linux or MacOS

In a terminal; fix the permissions on a single file by adding write permissions for the user (u) and the users' user group (g) with the chmod command. To fix the permissions on a directory and its files and subdirectories, add the --recursive option Such as:

The chmod command can also be appended to the script of a pipeline step if the permissions issue occurs with every pipeline executed. For example:

Restart the runner by re-running the command shown when setting up the runner, or recreate the runner in Bitbucket. For information on setting up the runner, see: Set up runners for Linux.

Accessing the log files for Windows PowerShell runners

1. Access the Windows device hosting the affected runner.

2. Using PowerShell or File Explorer, navigate to the runner installation directory (atlassian-bitbucket-pipelines-runner). This directory will be stored where PowerShell was working when the runner was started. For example: if PowerShell was working in the users' home directory, such as:

   1 PS C:\Users\Username>

   The runner installation directory would be C:\Users\Username\atlassian-bitbucket-pipelines-runner\

3. Open the runner.log file using a text editor. This log file is stored in the runners' working directory, in a subdirectory named using the runners' UUID (a 32-character hexadecimal string). Such as:

   1 <runner_working_directory>\<runnerUuid>\runner.log

   For example:

   1 .\atlassian-bitbucket-pipelines-runner\temp\b609961f-891e-c872-c36b-f3f2c315d186\runner.log

   The default working directory for the runner is the \temp subdirectory of the atlassian-bitbucket-pipelines-runner directory.

4. Review the end of the log file for errors that need to be addressed.

Fixing the permissions of a file or folder on Windows

To fix file or directory access permissions, either:

• Open the file Properties dialog from File Explorer and update the user permissions for the current user to Full Access for the files or folders preventing cleanup.

• Use the PowerShell Set-Acl command to provide write and delete access to the files or folders preventing cleanup. For information on using the Set-Acl command, visit Microsoft Docs — PowerShell Set-Acl cmdlet.

Restart the runner by re-running the command shown when setting up the runner, or recreate the runner in Bitbucket. For information on setting up the runner, see: Set up runners for Windows.