Pipelines : host key verification failed while cloning private repository with docker-in-docker

Summary

This document is to explain how to resolve the "Host key verification failed" error when cloning another private repository from Bitbucket pipelines with docker-in-docker.

Cause

• There might be a lot of scenarios where you have to clone another private repository from within a pipeline build. For instance, node or composer packages from another private repository are included in your current build, and during the build, these dependency repositories are cloned.

• If your command(that clones the repositories) is directly present in the bitbucket-pipelines.yml file, this document doesn't apply to you. For example, npm install command here has some dependencies to be downloaded from other private repositories.

  1 2 3 4 5 6 7 pipelines: branches: master: - step: script: - npm install

• In case you are using the docker-in-docker feature executing commands such as docker build or docker compose and including commands like git clone or npm install or yarn install or composer install in your configuration as shown in the example below, you might encounter the "host key verification failed" error.

  Sample bitbucket-pipelines.yml file:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 image: name: atlassian/default-image:4 pipelines: branches: main: - step: name: Build and Test Dev script: - echo "testing" - docker build -t testrepo/mytest:$BITBUCKET_BUILD_NUMBER . services: - docker

  Sample Dockerfile that's used in the above build:

  1 2 3 4 FROM ubuntu RUN apt-get install git -y RUN git clone git@bitbucket.org:myworkspace/mytestrepo.git

  • If you run the above build, you might encounter a "host key verification failed" error. This is because, in bitbucket pipelines, we automatically add a known_hosts file entry for bitbucket.org during the build setup phase in the build container. While you are running docker build or docker compose, you are using docker-in-docker here(docker service container). So the container that spun up by default doesn't have bitbucket.org host key entries. We automatically add these host keys in known_hosts only during the build setup in your build container and not the service containers spun up during the build.

Solution

To resolve this issue, you'll have to include a command to manually add the known_hosts file entries for bitbucket.org before the command that clones the other private repositories.

Example: We used docker build command in this case. So the Dockerfile would look like,

The new RUN instructions are to fetch the host keys of bitbucket.org and add them to your docker build.