// Article - How to include Load Testing in your CI / CD pipeline?
Written January 19, 2021
What is a CI/CD pipeline?
In software engineering, CI/CD or CICD generally refers to the combined practices of continuous integration and either continuous delivery or continuous deployment.
CI/CD bridges the gaps between development and operation activities and teams by enforcing automation in building, testing and deployment of applications. Modern day DevOps practices involve continuous development, continuous testing, continuous integration, continuous deployment and continuous monitoring of software applications throughout its development life cycle. The CI/CD practice or CI/CD pipeline forms the backbone of modern day DevOps operations.
Continuous integration
Developers practicing continuous integration merge their changes back to the main branch as often as possible. The developer's changes are validated by creating a build and running automated tests against the build. By doing so, you avoid integration challenges that can happen when waiting for release day to merge changes into the release branch.
Continuous integration puts a great emphasis on testing automation to check that the application is not broken whenever new commits are integrated into the main branch.
Continuous delivery
Continuous delivery is an extension of continuous integration since it automatically deploys all code changes to a testing and/or production environment after the build stage.
This means that on top of automated testing, you have an automated release process and you can deploy your application any time by clicking a button.
In theory, with continuous delivery, you can decide to release daily, weekly, fortnightly, or whatever suits your business requirements. However, if you truly want to get the benefits of continuous delivery, you should deploy to production as early as possible to make sure that you release small batches that are easy to troubleshoot in case of a problem.
Continuous deployment
Continuous deployment goes one step further than continuous delivery. With this practice, every change that passes all stages of your production pipeline is released to your customers. There's no human intervention, and only a failed test will prevent a new change to be deployed to production.
Continuous deployment is an excellent way to accelerate the feedback loop with your customers and take pressure off the team as there isn't a Release Day anymore. Developers can focus on building software, and they see their work go live minutes after they've finished working on it.
source = Atlassian CI/CD
How do I create a test workflow?
A test workflow is a collection of API calls in the form of steps that define the actions to be performed by the Rungutan Load Testing platform in order to simulate your desired scenarios.
A test workflow must have at least 1 step is defined using a JSON templating logic through either our JSON Web Editor (which also supports syntax highlighting) or through your preferred local Text Editor.
Since our platform was build in order to allow extensibility and accessibility of data through either APIs or via WEB access, you can use your JSON template through the CLI, your preffered CI/CD system or through the web platform. The result will always be exactly the same!
Sample workflow
Below you'll find the definition of a sample API which uses JWT for its authentication mechanism:
{
"test_name": "Platform with JWT based auth",
"num_clients": 10,
"run_time": 60,
"threads_per_region": 1,
"test_region": [ "us-east-1" ],
"workflow": [
{
"path": "https://example.com/login_path",
"method": "POST",
"headers": {
"Content-Type": "application/x-www-form-urlencoded"
},
"data": "user=user&password=pass",
"extract": [
{
"parameter_name": "authtoken",
"location": "body",
"key": "access_token"
}
]
},
{
"path": "https://example.com/my/profile",
"method": "GET",
"data": "",
"headers": {
"Authorization": "Bearer ${authtoken}"
}
},
{
"path": "https://example.com/results",
"method": "POST",
"data": "{\"result_id\": \"1\"}"
"headers": {
"Authorization": "Bearer ${authtoken}"
}
}
]
}
Using the workflow logic defined above, we have instructed the Rungutan Load Testing platform to:
- Perform 3 API calls, in order, towards the following paths: "/login_path" through POST, "/my/profile" through GET and "/results" through POST
- The parameter "access_token" is extracted after each and every call made in the first step from the respective JSON response body and stored as a parameter within the workflow with the name "authtoken"
- The previously saved value within the parameter "authtoken" is after that reused as part of the "Authorization" header in the subsequent workflow steps
Workflow used in demo
Within the demo we actually tested our own platform.
Here's the code used and explanation of its entire configuration:
{
"num_clients": 1,
"run_time": 10,
"threads_per_region": 1,
"workflow": [
{
"path": "https://example.com/v1/api/tests/list",
"method": "POST",
"data": "{\"team_id\":\"rungutan\"}",
"headers": {
"X-Api-Key": "${vault.api_key}",
"content-type": "application/json"
},
"extract": [
{
"parameter_name": "testId",
"location": "body",
"key": "Tests.0.test_id"
}
]
},
{
"path": "https://example.com/v1/api/tests/get",
"method": "POST",
"data": "{\"test_id\": \"${testId}\",\"team_id\": \"rungutan\"}",
"headers": {
"X-Api-Key": "${vault.api_key}",
"content-type": "application/json"
},
"extract": []
}
],
"test_region": [
"us-east-1"
],
"test_name": "DEMO - slow test with only successful results"
}
Using the workflow logic defined above, we have instructed the Rungutan Load Testing platform to:
- Perform 2 API calls, in order, towards the following paths: "/v1/api/tests/list" through POST and "/v1/api/tests/get" through POST
- The parameter "Tests.0.test_id" (meaning that specifically the "property test_id of the 1st item within the array Tests") is extracted after each and every call made in the first step from the respective JSON response body and stored as a parameter within the workflow with the name "testId"
- The previously saved value within the parameter "testId" is after that reused as part of the payload in the subsequent step
- The "${vault.api_key}" specification is a method of storing and sharing parameters and secrets with the members of your team throughout any tests so that they can re-use it (without having to know the actual value); more info on this here -> Documentation - Vault
If you're interested in seeing a few samples, you can check it out in our Documentation - Sample Workflows.
What CI/CD systems does Rungutan support?
Due to our platform’s ability to understand and process API calls from either a CLI, an API call or actions on the WebPlatform, it was extremely easy to offer you guys integrations to pretty much anything.
The list of actual Integrations is pretty much limitless, all needing to happen for that is simply reusing the CLI or provided Docker container to do your job.
However, here’s a short summary of implementations for different CI/CD systems.
GitLab pipelines
image: "python:3.7-alpine"
stages:
- load_test
variables:
RUNGUTAN_TEAM_ID: your_team
RUNGUTAN_API_KEY: your_api_key
before_script:
- pip install rungutan
load_test:
stage: load_test
script:
- rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${CI_PROJECT_PATH_SLUG}-${CI_PIPELINE_ID}
Jenkins
def RUNGUTAN_TEAM_ID=your_team
def RUNGUTAN_API_KEY=your_api_key
pipeline {
agent any
stages {
stage('LoadTest') {
agent {
docker {
image 'rungutancommunity/rungutan-cli:latest'
args '-u root -e ${RUNGUTAN_TEAM_ID} -e ${RUNGUTAN_API_KEY}'
reuseNode true
}
}
steps {
script {
rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${BUILD_TAG}
}
}
}
}
}
GitHub Actions
name: Load test with Rungutan
on:
release:
types:
- created
jobs:
load:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Load test your platform with Rungutan
uses: Rungutan/[email protected]
env:
RUNGUTAN_TEAM_ID: ${{ secrets.RUNGUTAN_TEAM_ID }}
RUNGUTAN_API_KEY: ${{ secrets.RUNGUTAN_API_KEY }}
RUNGUTAN_TEST_FILE: test_file.json
RUNGUTAN_TEST_NAME: ${{ github.repository }}-${{ github.ref }}
How do I set up a GitHub workflow?
A workflow is a configurable automated process made up of one or more jobs. You must create a YAML file to define your workflow configuration.
GitHub Actions is available with GitHub Free, GitHub Pro, GitHub Free for organizations, GitHub Team, GitHub Enterprise Cloud, GitHub Enterprise Server, and GitHub One. GitHub Actions is not available for private repositories owned by accounts using legacy per-repository plans.
Workflow files use YAML syntax, and must have either a .yml or .yaml file extension. If you're new to YAML and want to learn more, see Learn YAML in five minutes.
You must store workflow files in the .github/workflows directory of your repository.
GitHub also has a Marketplace which contains a lot of free tools to help you improve your workflow. One of them is of course -> Rungutan
In order to set up a GitHub workflow, all you have to do is create a folder called .github and then follow the below structure of files to define your workflow locations:
~/.github (main) $ tree
.
└── workflows
└── main.yml
1 directory, 1 file
In order to use our plugin, you have to define a set of variables and a set of secrets, as mentioned of course on the plugin's webpage.
GitHub secrets are a set of encrypted (and obfuscated) strings that are defined (usually) on a per-repository basis which contain sensitive informations to be used as part of the GitHub Actions workflows.
In our case, these are:
- RUNGUTAN_TEAM_ID - ~Required option~ - The team name which you'll be running the test for (displayed in the top left corner of your dashboard)
- RUNGUTAN_API_KEY - ~Required option~ - Your personal Rungutan API Key which you can obtain by clicking on the "Copy API key" button in the top left corner of your dashboard
Other than that, there are a few other options that you should check out:
- RUNGUTAN_TEST_FILE - ~Required option~ - The filename in your repository which contains the test case
- RUNGUTAN_TEST_NAME - ~Optional~ - If not provided, script will check for value of "test_name in your test case file
- RUNGUTAN_WAIT_FINISH - ~Optional~ - If not provided, the script will simply launch the test but not wait for it to finish
So here's how our pipeline looks like:
~/.github (main) $ cat workflows/main.yml
name: Load test with Rungutan
on:
# Triggers the workflow on push events but only for the "main" branch
push:
branches: [ main ]
jobs:
load:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Load test your platform with Rungutan
uses: Rungutan/[email protected]
env:
RUNGUTAN_TEAM_ID: ${{ secrets.RUNGUTAN_TEAM_ID }}
RUNGUTAN_API_KEY: ${{ secrets.RUNGUTAN_API_KEY }}
RUNGUTAN_TEST_FILE: test_file.json
RUNGUTAN_TEST_NAME: ${{ github.repository }}-${{ github.ref }}
Final thoughts
The reason why we wrote the article is that we're trying to raise awareness on why Load Testing should be performed after each and every release.
The logic behind this assumption is pretty straight forward -> each and every code or infrastructure change has the potential to either optimize or downgrade the responsiveness of your platform, which in turn affects of course your users' experience.
That is why load testing after each and every release will provide you an accurate overall of how your platform is running and how it feels like for your customers.
If you need any help implementing this, we're more than welcome to assist you through either our built-in chat system (through Intercom), the below contact form or through email.
Also, if you enjoyed this article, don't forget to Like and Subscribe to our YouTube channel -> Rungutan Load Testing.
Share this article on social media:
