• Products
  • Documentation
  • Resources

Deploy ImageLabeller with Github

To demonstrate how to develop, deploy, and manage applications using Jira Software and various connected tools, our team created ImageLabeller, a simple demo application built on AWS that uses machine learning to apply labels to images.

This page covers how to deploy ImageLabeller with Github. Before you begin, we recommend reading the ImageLabeller architecture and AWS SageMaker setup pages for context.

Prerequisites

If you don’t already have a Github organization, follow the steps in this Github guide to create one from scratch.

Public facing Github repositories with ImageLabeller code

https://github.com/AtlassianOpenDevOpsGuides

Integrate Jira and GitHub

From Jira click Board, then Apps, then GitHub.

screenshot of "github" highlighted in jira's app menu

Click Get it now.

screenshot of github app summary in jira

Click Apps, then Manage apps, and expand GitHub.

screenshot of manage apps page in jira

Click Get started.

screenshot of user-installed apps in jira

Click Get started.

screenshot of github configuration process in jira

Install Jira plugin in GitHub

Click Install Jira to install the Jira plugin to GitHub. This step is necessary if the Jira plugin is not already installed in GitHub.

screenshot of jira plugin installation in github

Click on your organization. In this example PmmQuickStartGuides01.

screenshot of jira installation process in jira

Click Install.

screenshot of installing jira in github part 3

Add GitHub organization to Jira

Go to Jira, click Apps, then Manage Apps, expand GitHub, and click Get started again.

screenshot of manage apps options in jira

Click Add an organization again. You will see your GitHub organization this time. Click Install.

screenshot of connecting an organization in jira after plugin installation

Jira will start to sync data from GitHub.

sceenshot of github configuration in progress

When the sync is complete the Sync Status changes to COMPLETE.

screenshot of github configuration complete

Setup SSH access

Set up GitHub SSH keys for your account by following the instructions here. Additional reference material is present here.

Setup a Personal Access Token

Setup a Personal Access Token by following this guide. The PAT is used to clone the SystemTest repo during integration testing steps. Give the PAT repo and workflow access.

Create a repository for AWS S3 infrastructure

A standard developer loop typically has a developer picking up a task from Jira, moving it to work in progress, and then doing the development work. The Jira issue ID is the key which ties the development work to the Jira issue. It is the core integration component between the two systems.

Go to Jira, and create a new issue for adding an AWS S3 infrastructure repository to GitHub. Make note of the issue ID. IM-9 in this example.

screenshot of jira issue to create repository "s3_infra"

Go to GitHub and click New. Choose the appropriate organization for Owner. Click Create repository to proceed.

screenshot of creating repository "s3_infra" in github

Add an AWS access key repository variables

Click Settings, then Secrets. Click New repository secret, and add AWS access key ID, and AWS secret access key. Give the IAM user associated with the AWS access key AdministratorAccess. You can opt to use more fine grained access control by choosing individual AWS access policies, but the details are left to the reader.

screenshot of adding AWS access key

In your terminal go to your s3_infra repository, and run the following to push to GitHub.

1 2 3 4 5 git add --all git commit -m "IM-9 add s3_infra repository to github" git remote add origin git@github.com:PmmQuickStartGuides01/s3_infra.git git branch -m mainline git push -u origin mainline

GitHub actions for deploying to AWS

Go to your s3_infra repository in your terminal, create a branch named after your Jira issue ID, and create a .github/workflows directory.

1 2 git checkout -b IM-9 mkdir -p .github/workflows && cd .github/workflows

Create deploy-test-staging.yml with the following yaml in the new .github/workflows directory. This defines a deployment workflow for your Test, and Staging environments that runs during pushes to branches other than mainline.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 name: deploy-s3-infra-test-staging on: push: branches: - '*' - '!mainline' jobs: deploy-us-west-1: runs-on: ubuntu-latest outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Checkout code uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-1" - name: Deploy to AWS CloudFormation uses: aws-actions/aws-cloudformation-github-deploy@v1 with: name: OpenDevOpsS3Infra template: template.yml no-fail-on-empty-changeset: "1" deploy-us-east-2: runs-on: ubuntu-latest needs: deploy-us-west-1 steps: - name: Checkout code uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-2" - name: Deploy to AWS CloudFormation uses: aws-actions/aws-cloudformation-github-deploy@v1 with: name: OpenDevOpsS3Infra template: template.yml no-fail-on-empty-changeset: "1"

Then create deploy-prod.yml with the following yaml. This defines a deployment workflow for your Production environments that runs when a pull request merges changes into mainline.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 name: deploy-s3-infra-prod on: pull_request: branches: - mainline jobs: deploy-us-west-2: runs-on: ubuntu-latest outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Checkout code uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-2" - name: Deploy to AWS CloudFormation uses: aws-actions/aws-cloudformation-github-deploy@v1 with: name: OpenDevOpsS3Infra template: template.yml no-fail-on-empty-changeset: "1" deploy-ca-central-1: runs-on: ubuntu-latest needs: deploy-us-west-2 steps: - name: Checkout code uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "ca-central-1" - name: Deploy to AWS CloudFormation uses: aws-actions/aws-cloudformation-github-deploy@v1 with: name: OpenDevOpsS3Infra template: template.yml no-fail-on-empty-changeset: "1" deploy-us-east-1: runs-on: ubuntu-latest needs: deploy-ca-central-1 steps: - name: Checkout code uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-1" - name: Deploy to AWS CloudFormation uses: aws-actions/aws-cloudformation-github-deploy@v1 with: name: OpenDevOpsS3Infra template: template.yml no-fail-on-empty-changeset: "1"

Understanding GitHub actions

Triggers

Declare events that trigger the workflow at the top of these files. In deploy-test-staging.yml the event is pushes to all branches except mainline.

1 2 3 4 5 6 name: deploy-s3-infra-test-staging on: push: branches: - '*' - '!mainline'

In deploy-prod.yaml the event is pull requests to mainline branch.

1 2 3 4 5 name: deploy-s3-infra-prod on: pull_request: branches: - mainline

Many events can trigger workflows. See documentation here for more information.

Jobs

A workflow contains many jobs that run when the triggering event occurs. Each job has a set of steps that run when the job is executed. There is a step to checkout the repository code, a step to configure AWS credentials, and a step to deploy to AWS using AWS CloudFormation in this example.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 jobs: deploy-us-west-1: runs-on: ubuntu-latest outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Checkout code uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-1" - name: Deploy to AWS CloudFormation uses: aws-actions/aws-cloudformation-github-deploy@v1 with: name: OpenDevOpsS3Infra template: template.yml no-fail-on-empty-changeset: "1"

Read more about jobs here, and more about steps here.

Enforcing order with needs

You can specify an order to your jobs using needs. GitHub runs all steps in parallel by default. Use needs to make one step depend on another.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 deploy-us-east-1: runs-on: ubuntu-latest needs: deploy-ca-central-1 steps: - name: Checkout code uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-1" - name: Deploy to AWS CloudFormation uses: aws-actions/aws-cloudformation-github-deploy@v1 with: name: OpenDevOpsS3Infra template: template.yml no-fail-on-empty-changeset: "1"

Pushing to a feature branch

From the command line run the following to push your code to the IM-9 branch of your s3_infra repository.

1 2 3 git add --all git commit -m "IM-9 add GitHub actions to s3_infra" git push -u origin IM-9

Click Actions to see running workflows.

screenshot of running workflows in github

Creating a pull request

Click Pull requests followed by Create pull request.

screenshot of changes in github

Choose your feature branch as the source branch, then click Create pull request.

screenshot of open pull request in github

Review the code changes, then click Merge pull request to accept the changes.

screenshot of code changes in github

Click Actions to see that the Production deployment has started. GitHub runs the jobs in deploy-prod.yml because the branch is mainline.

screenshot of production deployment in github

Create a repository for SubmitImage AWS Lambda

Go to Jira and create a Jira issue for adding a SubmitImage AWS Lambda repository to GitHub. Take note of the Jira issue ID. IM-8 in this example.

screenshot of adding jira issue to create repository "submitimage" in github

Go to GitHub and click New. Choose the appropriate organization for Owner. Click Create repository to proceed.

screenshot of creating repository "submitimage" in github

Click Settings, then Secrets. Add your Personal Access Token as ACCESS_KEY, your AWS access key id as AWS_ACCESS_KEY_ID, your AWS secret access key as AWS_SECRET_ACCESS_KEY, and your AWS account ID as AWS_ACCOUNT_ID.

screenshot of adding personal access token in github

In your terminal go to your SubmitImage repository, and run the following to push your code to GitHub.

1 2 3 4 5 git add --all git commit -m "IM-8 add SubmitImage to github" git remote add origin git@github.com:PmmQuickStartGuides01/submitImage.git git branch -m mainline git push -u origin mainline


GitHub actions for deploying to AWS

Go to your SubmitImage repository in your terminal, create a branch named after your Jira issue ID, and create a .github/workflows directory.

1 2 git checkout -b IM-8 mkdir -p .github/workflows && cd .github/workflows

Create deploy-test-staging.yml with the following yaml in the new .github/workflows directory. This defines a deployment workflow for your Test, and Staging environments that runs during pushes to branches other than mainline.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 name: deploy-submit-image-test-staging on: push: branches: - '*' - '!mainline' env: aws_account_id: ${{secrets.AWS_ACCOUNT_ID}} jobs: run-unit-tests: runs-on: ubuntu-latest steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Checkout code uses: actions/checkout@v2 - name: Test run: | cd ${{ github.workspace }}/submitImage ls go test ./opendevopslambda... deploy-us-west-1: runs-on: ubuntu-latest needs: run-unit-tests outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-1" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsSubmitImage" \ --s3-bucket "open-devops-code-us-west-1-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-us-west-1: runs-on: ubuntu-latest needs: deploy-us-west-1 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-1" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=us-west-1 deploy-us-east-2: runs-on: ubuntu-latest needs: integration-test-us-west-1 outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-2" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsSubmitImage" \ --s3-bucket "open-devops-code-us-east-2-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-us-east-2: runs-on: ubuntu-latest needs: deploy-us-east-2 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-2" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=us-east-2

Then create deploy-prod.yml with the following yaml. This defines a deployment workflow for your Production environments that runs when a pull request merges changes into mainline.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 name: deploy-submit-image-prod on: pull_request: branches: - mainline env: aws_account_id: ${{secrets.AWS_ACCOUNT_ID}} jobs: deploy-us-west-2: runs-on: ubuntu-latest outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-2" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsSubmitImage" \ --s3-bucket "open-devops-code-us-west-2-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-us-west-2: runs-on: ubuntu-latest needs: deploy-us-west-2 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-2" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=us-west-2 deploy-us-east-1: runs-on: ubuntu-latest needs: integration-test-us-west-2 outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-1" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsSubmitImage" \ --s3-bucket "open-devops-code-us-east-1-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-us-east-1: runs-on: ubuntu-latest needs: deploy-us-east-1 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-1" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=us-east-1 deploy-ca-central-1: runs-on: ubuntu-latest needs: integration-test-us-east-1 outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "ca-central-1" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsSubmitImage" \ --s3-bucket "open-devops-code-ca-central-1-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-ca-central-1: runs-on: ubuntu-latest needs: deploy-ca-central-1 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "ca-central-1" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=ca-central-1

Understanding GitHub actions

This job uses AWS SAM to deploy your SubmitImage AWS Lambda to us-west-2.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 deploy-us-west-2: runs-on: ubuntu-latest outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-2" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsSubmitImage" \ --s3-bucket "open-devops-code-us-west-2-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

This job clones the SystemTests repository, and runs integrations tests in us-west-2.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 integration-test-us-west-2: runs-on: ubuntu-latest needs: deploy-us-west-2 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-2" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=us-west-2

This step uses the Personal Access Token to clone the SystemTests repository.

1 2 3 4 5 - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }}

Pushing to a feature branch

Run the following from the command line to push your changes to the IM-8 branch of your SubmitImage repository. Include the Jira issue ID in commit messages, and branch names to enable the Jira GitHub integration to keep track of what is happening in your project.

1 2 3 git add --all git commit -m "IM-8 add github actions to SubmitImage" git push -u origin IM-8

Click Actions to see running workflows.

screenshot of running workflows in github

Notice that the unit test job has passed, and deployment to Test us-west-1 has started.

screenshot of running workflows started in github

The system tests defined earlier are run as part of the integration-test-us-west-1, and integration-test-us-east-2 jobs.

screenshot of running workflows in porgress

Create a pull request

To create a pull request click Pull requests, then New pull request.

screenshot of pull requests in github

Choose to merge from your feature branch.

screenshot of branch options in github

Click Create pull request.

screenshot of creating pull request in github

Merge the pull request, and delete the feature branch. Click Actions to monitor the Production deployment.

screenshot of monitoring deployment in github

Create a repository for InvokeLabeller AWS Lambda

Go to Jira and create a Jira issue for adding a InvokeLabeller AWS Lambda repository to GitHub. Take note of the Jira issue ID. IM-11 in this example.

screenshot of jira issue to create repository "invokelabeller"

Go to GitHub and click New. Choose the appropriate organization for Owner. Click Create repository to proceed.

screenshot of creating repository "invokelabeller" in github

Click Settings, then Secrets. Add your Personal Access Token as ACCESS_KEY, your AWS access key id as AWS_ACCESS_KEY_ID, your AWS secret access key as AWS_SECRET_ACCESS_KEY, and your AWS account ID as AWS_ACCOUNT_ID.

screenshot of adding personal access token in github

In your terminal go to your InvokeLabeller repository, and run the following to push your code to GitHub.

1 2 3 4 5 git add --all git commit -m "IM-11 add InvokeLabeller to github" git remote add origin git@github.com:PmmQuickStartGuides01/InvokeLabeller.git git branch -m mainline git push -u origin mainline

GitHub actions for deploying to AWS

Go to your InvokeLabeller repository in your terminal, create a branch named after your Jira issue ID, and create a .github/workflows directory.

1 2 git checkout -b IM-11 mkdir -p .github/workflows && cd .github/workflows

Create deploy-test-staging.yml with the following yaml in the new .github/workflows directory. This defines a deployment workflow for your Test, and Staging environments that runs during pushes to branches other than mainline.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 name: deploy-submit-image-test-staging on: push: branches: - '*' - '!mainline' env: aws_account_id: ${{secrets.AWS_ACCOUNT_ID}} jobs: run-unit-tests: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v2 - uses: actions/setup-python@v2 with: python-version: '3.x' - name: Test run: | pip3 install pytest pip3 install moto pip3 install -r tst/requirements.txt --user python3 -m pytest -v tst/unit --junitxml=test-reports/report.xml deploy-us-west-1: runs-on: ubuntu-latest needs: run-unit-tests outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-1" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsImageLabeller" \ --s3-bucket "open-devops-code-us-west-1-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-us-west-1: runs-on: ubuntu-latest needs: deploy-us-west-1 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-1" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=us-west-1 deploy-us-east-2: runs-on: ubuntu-latest needs: integration-test-us-west-1 outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-2" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsImageLabeller" \ --s3-bucket "open-devops-code-us-east-2-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-us-east-2: runs-on: ubuntu-latest needs: deploy-us-east-2 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-2" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=us-east-2

Then create deploy-prod.yml with the following yaml. This defines a deployment workflow for your Production environments that runs when a pull request merges changes into mainline.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 name: deploy-submit-image-prod on: pull_request: branches: - mainline env: aws_account_id: ${{secrets.AWS_ACCOUNT_ID}} jobs: deploy-us-west-2: runs-on: ubuntu-latest outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-2" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsImageLabeller" \ --s3-bucket "open-devops-code-us-west-2-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-us-west-2: runs-on: ubuntu-latest needs: deploy-us-west-2 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-west-2" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=us-west-2 deploy-us-east-1: runs-on: ubuntu-latest needs: integration-test-us-west-2 outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-1" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsImageLabeller" \ --s3-bucket "open-devops-code-us-east-1-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-us-east-1: runs-on: ubuntu-latest needs: deploy-us-east-1 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "us-east-1" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=us-east-1 deploy-ca-central-1: runs-on: ubuntu-latest needs: integration-test-us-east-1 outputs: env-name: ${{ steps.env-name.outputs.environment }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - uses: actions/checkout@v2 - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "ca-central-1" - name: SAM Validate run: | sam --version sam validate - name: SAM Build run: | sam build - name: SAM Deploy continue-on-error: true run: | sam deploy --template-file .aws-sam/build/template.yaml \ --stack-name "OpenDevOpsImageLabeller" \ --s3-bucket "open-devops-code-ca-central-1-${aws_account_id}" \ --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM integration-test-ca-central-1: runs-on: ubuntu-latest needs: deploy-ca-central-1 steps: - name: Pull systemTests repo uses: actions/checkout@master with: repository: PmmQuickStartGuides01/systemTests token: ${{ secrets.ACCESS_KEY }} - name: Configure AWS credentials id: creds uses: aws-actions/configure-aws-credentials@v1 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: "ca-central-1" - name: Install Go uses: actions/setup-go@v2 with: go-version: 1.16.x - name: Run integration tests run: go test -v ./... -aws_region=ca-central-1

Pushing to a feature branch

Run the following from the command line to push your changes to the IM-11 branch of your InvokeLabeller repository. Include the Jira issue ID in commit messages, and branch names to enable the Jira GitHub integration to keep track of what is happening in your project.

1 2 3 git add --all git commit -m "IM-11 add github actions to InvokeLabeller" git push -u origin IM-11

Click Actions to see running workflows. The system tests defined earlier are run as part of the integration-test-us-west-1, and integration-test-us-east-2 jobs.

screenshot of running workflows for invoke labeller

Create a pull request

To create a pull request click Pull requests, then New pull request. Choose to merge from your feature branch.

screenshot of comparing changes in github

Click Create pull request. Wait for all checks to pass then merge the pull request, and delete the feature branch.

screenshot of creating pull request in github

Click Actions to monitor the Production deployment.

screenshot of deployments in github

Create a repository for SystemTests

Go to Jira and create a Jira issue for adding a SystemTests repository to GitHub. Take note of the Jira issue ID. In this example it is IM-7.

screenshot of adding jira issue to create repository "systemtests"

Go to GitHub and click New. Choose the appropriate organization for Owner. Click Create repository to proceed.

screenshot of creating repository "systemtests" in github

In your terminal go to your SystemTests repository, and run the following to push your code to GitHub.

1 2 3 4 5 git add --all git commit -m "IM-7 add SystemTests repository to GitHub" git remote add origin git@github.com:PmmQuickStartGuides01/systemTests.git git branch -M mainline git push -u origin mainline

The SystemTests repository doesn’t need GitHub actions. It has no pipeline of its own since it provides tests for other pipelines to run. The integration test steps of the CI/CD workflow files can be uncommented, committed, and pushed once all components of the ImageLabeller application are deployed. The system tests will run them for the first time once the updated CI/CD workflow files are pushed. The tests will only pass if all components of the application are working properly.

If you’ve made this far, congratulations! You just deployed ImageLabeller. The next step is to set up monitoring ImageLabeller with Opsgenie.

 

Last modified on Sep 24, 2021
Cached at 3:23 AM on Oct 19, 2021 |

Additional Help