Hey Robot, run my tests!

written by Tatjana Statescu

Recently I shared with you how we use smoke testing as a pillar of our QA strategy.

We organize our automatic tests logically into test suites and run them as part of the pipelines, triggered by code push and merge hooks.

Often though, we would like to run a test suite on-demand, without having to bump code.

We can simply check out the test repository locally, and run tests from the local machine. However, that is both time-consuming and error-prone.

We thought it would be better if we could run test suites on-demand from Gitlab CI. We decided we would like to trigger tests from:

• the Gitlab user interface
• another CI pipeline
• a Slack channel

In this article, I describe how we set up those three, using our smoke tests as a prime example.

If you like this kind of QA automation content and are considering a new role, please check out our current jobs.

Our smoke tests are located in their own Gitlab repository, and we already had a Gitlab CI job to run them.

To run CI jobs in the correct environment, we pass a variable called ENVIRONMENT, which we use in a couple of places:

• In Gitlab CI Settings we set up environment variable values for each deployment target — it’s especially useful for storing protected parameters and environment settings.
• Our tests contain some conditional logic based on the variable value.

In the first case, we wanted to make it possible to trigger the test suite on-demand. So we created a dynamic pipeline that extends our pre-defined .job-system-smoke-test job.

As you can see below, we use the ENVIRONMENT variable value to inform Gitlab CI which environment to run in.

Now we can very easily trigger our smoke-test job directly from the Gitlab CI user interface. It prompts us to type in an ENVIRONMENT variable.

For our second use case, we aimed to trigger these tests from the pipeline of another project repository. To achieve this we added a rule in our test pipeline:

Then in the desired project repository .gitlab-ci.yml the file we created a job to trigger the smoke test pipeline, as follows:

You can read more about Gitlab multi-project pipelines here.

Our third use case is concerned with triggering tests directly from Slack. This could save us the annoyance of opening a browser and entering variables every time. Slack is our main communication tool, so why not play with Chat Ops and trigger the job directly from the Slack window.

Firstly, we needed to configure Slack and Gitlab CI. We created a special Slack channel for this purpose and installed a Gitlab CI app for Slack by clicking “More” and then “Add apps” in the Channel Details screen.

In Gitlab settings → Integrations we added the Slack application. We also needed to enter the project alias to call the tests. In our case, it’s “aurena-tech/system-smoke-tests”. More information on how to do that can be found in GitLab documentation.

In short, the GitLab app ChatOps functionality allows the execution of jobs only from the master branch or branch that is set default in the repository settings, and extra job arguments that are passed in the Slack call are packed into the CHAT_INPUT variable. For that we need to form our call this way:

We use <job arguments> to pass the ENVIRONMENT variable to the test job.

That took care of the user command input. After that, we needed something with Gitlab CI to interpret our commands and map them to the jobs.

For that, we created an extra pipeline for triggering smoke tests from the chat, as you can see:

We also had to adapt the job to process the CHAT_INPUT variable.

And that was it! Now we could trigger our tests from the Slack channel.

Typing in the Slack channel /gitlab aurena-tech/system-smoke-tests run system:smoke:test:chat development

The app returns a reply into the Slack room with the job output log.

Here is our combined .gitlab-ci.yml file

We are delighted with the results:

• More inspection and productivity. Every team member, no matter their technical know-how, can now trigger tests as required and see the results in one place.
• Code Reuse. Our CI/CD pipeline can trigger system or end-to-end tests from its repository by using Gitlab multi-project functionality.

We look forward to more experimentation with Gitlab Chat Ops functionality.