• bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
Job openings (5)

09.08.2021

Hey Robot, run my tests!

Learn how we run test suites on-demand from Gitlab CI, and how we trigger them from different pipelines and from Slack.

written by Tatjana Statescu

Recently I shared with you how we use smoke testing as a pillar of our QA strategy.

We organize our automatic tests logically into test suites and run them as part of the pipelines, triggered by code push and merge hooks.

Often though, we would like to run a test suite on-demand, without having to bump code.

We can simply check out the test repository locally, and run tests from the local machine. However, that is both time-consuming and error-prone.

We thought it would be better if we could run test suites on-demand from Gitlab CI. We decided we would like to trigger tests from:

  • the Gitlab user interface
  • another CI pipeline
  • a Slack channel

In this article, I describe how we set up those three, using our smoke tests as a prime example.

If you like this kind of QA automation content and are considering a new role, please check out our current jobs.

Setting up for multiple environments

Our smoke tests are located in their own Gitlab repository, and we already had a Gitlab CI job to run them.

To run CI jobs in the correct environment, we pass a variable called ENVIRONMENT, which we use in a couple of places:

  • In Gitlab CI Settings we set up environment variable values for each deployment target — it’s especially useful for storing protected parameters and environment settings.
  • Our tests contain some conditional logic based on the variable value.
Fragment from .gitlab-ci.yml
                .job-system-smoke-test:
  stage: system-smoke-tests
  script:
    - echo "Running tests in $ENVIRONMENT environment"
    - npm ci
    - npm run test-smoke
  artifacts:
    reports:
      junit: junit.xml
            

Triggering tests from Gitlab UI

In the first case, we wanted to make it possible to trigger the test suite on-demand. So we created a dynamic pipeline that extends our pre-defined .job-system-smoke-test job.

As you can see below, we use the ENVIRONMENT variable value to inform Gitlab CI which environment to run in.

Fragment from .gitlab-ci.yml
                system:smoke:test:dynamic:
  extends:
    - .job-system-smoke-test  
  environment:
    name: $ENVIRONMENT 
  rules:
    # For running the job from Gitlab UI ('Run Pipeline' in CI/CD section)
    - if: '$CI_PIPELINE_SOURCE == "web" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'
            

Now we can very easily trigger our smoke-test job directly from the Gitlab CI user interface. It prompts us to type in an ENVIRONMENT variable.

Triggering tests from another pipeline

For our second use case, we aimed to trigger these tests from the pipeline of another project repository. To achieve this we added a rule in our test pipeline:

Fragment from .gitlab-ci.yml
                system:smoke:test:dynamic:
  extends:
    - .job-system-smoke-test  
  environment:
    name: $ENVIRONMENT   
  rules:
    # For running the job from Gitlab UI ('Run Pipeline' in CI/CD section)
    - if: '$CI_PIPELINE_SOURCE == "web" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'  
    # For multi project pipelines
    - if: '$CI_PIPELINE_SOURCE == "pipeline" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'
            

Then in the desired project repository .gitlab-ci.yml the file we created a job to trigger the smoke test pipeline, as follows:

fragment of .gitlab-ci.yml
                .job-test-system:
  stage: system-tests
  variables:
    ENVIRONMENT: development
    RUN_SYSTEM_SMOKE_TEST: "TRUE"
  trigger:
    project: aurena-tech/system-smoke-tests
    branch: master
    strategy: depend
            

You can read more about Gitlab multi-project pipelines here.

Triggering test from Slack

Our third use case is concerned with triggering tests directly from Slack. This could save us the annoyance of opening a browser and entering variables every time. Slack is our main communication tool, so why not play with Chat Ops and trigger the job directly from the Slack window.

Firstly, we needed to configure Slack and Gitlab CI. We created a special Slack channel for this purpose and installed a Gitlab CI app for Slack by clicking “More” and then “Add apps” in the Channel Details screen.

In Gitlab settings → Integrations we added the Slack application. We also needed to enter the project alias to call the tests. In our case, it’s “aurena-tech/system-smoke-tests”. More information on how to do that can be found in GitLab documentation.

In short, the GitLab app ChatOps functionality allows the execution of jobs only from the master branch or branch that is set default in the repository settings, and extra job arguments that are passed in the Slack call are packed into the CHAT_INPUT variable. For that we need to form our call this way:

                /gitlab  <project name> <job name> <job arguments>
            

We use <job arguments> to pass the ENVIRONMENT variable to the test job.

That took care of the user command input. After that, we needed something with Gitlab CI to interpret our commands and map them to the jobs.

For that, we created an extra pipeline for triggering smoke tests from the chat, as you can see:

Fragment from .gitlab-ci.yml
                system:smoke:test:chat:
  extends:
    - .job-system-smoke-test  
  only: [chat]
  environment:
    name: $CHAT_INPUT
            

We also had to adapt the job to process the CHAT_INPUT variable.

Fragment from .gitlab-ci.yml
                .job-system-smoke-test:
  stage: system-smoke-tests
  before_script:
    - if [ -n "$CHAT_INPUT" ]; then export ENVIRONMENT=$CHAT_INPUT; fi
  script:
    - echo "Running tests in $ENVIRONMENT environment"
    - npm ci
    - npm run test-smoke
  artifacts:
    reports:
      junit: junit.xml
            

And that was it! Now we could trigger our tests from the Slack channel.

Typing in the Slack channel /gitlab aurena-tech/system-smoke-tests run system:smoke:test:chat development

The app returns a reply into the Slack room with the job output log.

All together

Here is our combined .gitlab-ci.yml file

.gitlab-ci.yml
                stages:
  - system-smoke-tests
 
# smoke test job definition
.job-system-smoke-test:
  stage: system-smoke-tests
  before_script:
    - if [ -n "$CHAT_INPUT" ]; then export ENVIRONMENT=$CHAT_INPUT; fi
  script:
    - echo "Running tests in $ENVIRONMENT environment"
    - npm ci
    - npm run test-smoke
  artifacts:
    reports:
      junit: junit.xml
 
# pipeline that runs smoke tests from Slack trigger
system:smoke:test:chat:
  extends:
    - .job-system-smoke-test  
  only: [chat]
  environment:
    name: $CHAT_INPUT
 
# pipeline that runs smoke tests from multi-project pipeline or from Gitlab UI
system:smoke:test:dynamic:
  extends:
    - .job-system-smoke-test  
  rules:
    # Never trigger for prod
    - if: '$ENVIRONMENT == "productive"'
      when: never
    # For multi project pipelines
    - if: '$CI_PIPELINE_SOURCE == "pipeline" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'
    # For running the job from Gitlab UI ('Run Pipeline' in CI/CD section)
    - if: '$CI_PIPELINE_SOURCE == "web" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'
            

Conclusion

We are delighted with the results:

  • More inspection and productivity. Every team member, no matter their technical know-how, can now trigger tests as required and see the results in one place.
  • Code Reuse. Our CI/CD pipeline can trigger system or end-to-end tests from its repository by using Gitlab multi-project functionality.

We look forward to more experimentation with Gitlab Chat Ops functionality.

Article by
Tatjana Statescu

Tatjana is QA lead at AURENA Tech. “The only thing better than finding a bug early is preventing it entirely”: With this in mind, Tatjana makes sure that quality is built into all development stages.

More articles

08.09.2022

Mobile application end-to-end test automation using Robot Framework

Teaching a robot to use a mobile phone: Learn how we write end-to-end tests for mobile applications.

25.08.2022

AURENA Tech Team Event 2022

The AURENA Tech Team Event 2022 brought together our team members from different European countries for two exciting days in Austria.

28.06.2022

Impressions of the AWS 2022 Berlin summit

Our DevOps Engineers report on their experience at the Amazon Web Services conference.

20.04.2022

Let's break the Vegas rule

Our Scrum Master GO shares her insights and main take-aways from the Atlassian Team ´22.

28.02.2022

AURENA Tech strengthens its DevOps team

We welcome Junior DevOps Engineer Danijel Maraz as the newest member to the AURENA Tech team.

Open positions

AURENA.tech
text
Node.js Developer (f/m/x)

In this role, you will work on feature development and continuous improvement of our leading real-time auction platform.

  • Leoben or fully remote
  • Fulltime, permanent
  • Starts at € 48,500 p.a.
AURENA.tech
text
Front-End Developer (HTML/CSS)

In this role, you will work on feature development and continuous improvement of our leading real-time auction platform.

  • Leoben or fully remote
  • Fulltime, permanent
  • Starts at € 44,800 p.a.
AURENA.tech
text
Senior React Developer (f/m/x)

Build new high-performance apps from scratch and develop new features for our leading real-time auction platform

  • Leoben or fully remote
  • Fulltime, permanent
  • Starts at € 60,200 p.a.
AURENA.tech
text
Senior Node.js Developer (f/m/x)

This role offers you the opportunity to lead middleware and microservice development at AURENA Tech.

  • Leoben or fully remote
  • Fulltime, permanent
  • Starts at € 60,200 p.a.
AURENA.tech
text
Senior Mobile App Developer (f/m/x)

Develop cross-platform native apps and progressive web apps from scratch.

  • Leoben or fully remote
  • Fulltime, permanent
  • Starts at € 60,200 p.a.