What are pipes?

Understanding pipes

Pipes provide a simple way to configure a pipeline. They are especially powerful when you want to work with third-party tools. Just paste the pipe into the yaml file, supply a few key pieces of information, and the rest is done for you. You can add as many pipes as you like to your steps, so the possibilities are endless!

1 2 3 4 5 6 7 8 script: - pipe: atlassian/aws-s3-deploy:0.2.2 variables: AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY AWS_DEFAULT_REGION: 'us-east-1' S3_BUCKET: 'my-bucket-name' LOCAL_PATH: 'build'

List of available pipes

  • You can find the complete list of pipes here.

  • If you need a custom approach to pipes you can create your own pipe.

How it works

A pipe uses a script that lives in a Docker container and has the commands you previously put in your yaml along with a few extras.

You'll only have to provide a few values. The provided pipes are public, so you can check the source code to see how it all works.


Example: add a pipe to upload to Amazon S3 bucket

If we want our pipeline to upload the contents of the build directory to our my-bucket-name S3 bucket, we can use the AWS S3 Deploy pipe.

AWS S3 deploy demo gif
  1. Select the AWS S3 Deploy pipe.

  2. Click Copy.

  3. Move the cursor to the script section of the deployment step (placement is important!). 

  4. Paste the copied code.

  5. Notice: We have removed the optional parameters, but you can uncomment them if you need them.

  6. Add the parameter values for our deployment (always in single quotes):

    1. using our predefined variables $AWS_ACCESS_KEY_ID and $AWS_SECRET_ACCESS_KEY

    2. specifying our region as us-east-1

    3. supplying the name of our bucket: my-bucket-name

    4. defining that we want to deploy the contents of our build folder

1 2 3 4 5 6 7 8 script: - pipe: atlassian/aws-s3-deploy:0.2.2 variables: AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY AWS_DEFAULT_REGION: 'us-east-1' S3_BUCKET: 'my-bucket-name' LOCAL_PATH: 'build'

For some deployment pipes, like AWS Elastic Beanstalk Deploy and NPM Publish, we also provide a convenient link in the logs to view the deployed application.

deployment example gif

How to use pipes

There are two ways to add pipes to your pipeline:

1. Use the online editor

  1. Open up your bitbucket-pipelines.yml file in the editor.

  2. Select the pipe you need.

  3. Copy the pipe, and paste it into the script section of your step.

  4. Add your specific values in single quotes (and un-comment any optional variables you want to use).

  5. Run your build.

You can fill in the variable values in-line, or use predefined variables.

2. Edit the configuration directly

You can add the details of the task to your bitbucket-pipelines.yml file using an editor of your choice.

The README.md file in the available pipes listed below contains instructions on how to use the pipe and the lines you can copy and paste into your bitbucket-pipelines.yml file. While you are in the pipe repo you can have a peek at the scripts to see all the good stuff the pipe is doing behind the scenes.

Considerations:

  • Pipes need to use the Docker service, which we start transparently. This counts towards the number of services you can run in a step. By default, this docker service can use 1GB of memory, but you can change that in your configuration if the pipe needs more memory.

  • Pipes use semantic versioning, so as we make changes to the pipe the version number will change. We'll be updating the repository changelog as we go. If you want to use a different version, you can by changing the version number in your bitbucket-pipelines.yml file. Note that  0.x.y versioned pipes are an exception to classic semantic versioning, as they are still under development, and can introduce breaking changes between minor and patch version changes.

    Example:

    changing pipe: atlassian/aws-s3-deploy:0.1.0

    to pipe: atlassian/aws-s3-deploy:0.2.0

Missing pipe?

If there is a pipe you'd like to see that we don't already have you can create your own pipe, or use the Suggest a pipe box in the Bitbucket editor.

image displaying the Suggest a pipe field and Send button

Need help with a pipe?

The quickest way to get help is to follow the pipe's support instructions, found in its repository's readme (also seen in the editor when you select a pipe).

Another option, for general help or if you have an issue or feature request, is to let us know on Community.

If you’re reporting an issue, you can speed up response times by including:

  • the version of the pipe

  • relevant logs and error messages

  • steps to reproduce

Additional Help