• bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
  • bids: 0 +10
  • bidders: 0 +10
  • completed auctions: 0 +10
  • next fall of the hammer: 0 ending now
  • sold lots: 0 +10
Job openings (5)

09.08.2021

Hey Robot, run my tests!

Learn how we run test suites on-demand from Gitlab CI, and how we trigger them from different pipelines and from Slack.

Recently I shared with you how we use smoke testing as a pillar of our QA strategy.

We organize our automatic tests logically into test suites and run them as part of the pipelines, triggered by code push and merge hooks.

Often though, we would like to run a test suite on-demand, without having to bump code.

We can simply check out the test repository locally, and run tests from the local machine. However, that is both time-consuming and error-prone.

We thought it would be better if we could run test suites on-demand from Gitlab CI. We decided we would like to trigger tests from:

  • the Gitlab user interface
  • another CI pipeline
  • a Slack channel

In this article, I describe how we set up those three, using our smoke tests as a prime example.

If you like this kind of QA automation content and are considering a new role, please check out our current jobs.

Setting up for multiple environments

Our smoke tests are located in their own Gitlab repository, and we already had a Gitlab CI job to run them.

To run CI jobs in the correct environment, we pass a variable called ENVIRONMENT, which we use in a couple of places:

  • In Gitlab CI Settings we set up environment variable values for each deployment target — it’s especially useful for storing protected parameters and environment settings.
  • Our tests contain some conditional logic based on the variable value.
Fragment from .gitlab-ci.yml
                .job-system-smoke-test:
  stage: system-smoke-tests
  script:
    - echo "Running tests in $ENVIRONMENT environment"
    - npm ci
    - npm run test-smoke
  artifacts:
    reports:
      junit: junit.xml
            

Triggering tests from Gitlab UI

In the first case, we wanted to make it possible to trigger the test suite on-demand. So we created a dynamic pipeline that extends our pre-defined .job-system-smoke-test job.

As you can see below, we use the ENVIRONMENT variable value to inform Gitlab CI which environment to run in.

Fragment from .gitlab-ci.yml
                system:smoke:test:dynamic:
  extends:
    - .job-system-smoke-test  
  environment:
    name: $ENVIRONMENT 
  rules:
    # For running the job from Gitlab UI ('Run Pipeline' in CI/CD section)
    - if: '$CI_PIPELINE_SOURCE == "web" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'
            

Now we can very easily trigger our smoke-test job directly from the Gitlab CI user interface. It prompts us to type in an ENVIRONMENT variable.

Triggering tests from another pipeline

For our second use case, we aimed to trigger these tests from the pipeline of another project repository. To achieve this we added a rule in our test pipeline:

Fragment from .gitlab-ci.yml
                system:smoke:test:dynamic:
  extends:
    - .job-system-smoke-test  
  environment:
    name: $ENVIRONMENT   
  rules:
    # For running the job from Gitlab UI ('Run Pipeline' in CI/CD section)
    - if: '$CI_PIPELINE_SOURCE == "web" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'  
    # For multi project pipelines
    - if: '$CI_PIPELINE_SOURCE == "pipeline" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'
            

Then in the desired project repository .gitlab-ci.yml the file we created a job to trigger the smoke test pipeline, as follows:

fragment of .gitlab-ci.yml
                .job-test-system:
  stage: system-tests
  variables:
    ENVIRONMENT: development
    RUN_SYSTEM_SMOKE_TEST: "TRUE"
  trigger:
    project: aurena-tech/system-smoke-tests
    branch: master
    strategy: depend
            

You can read more about Gitlab multi-project pipelines here.

Triggering test from Slack

Our third use case is concerned with triggering tests directly from Slack. This could save us the annoyance of opening a browser and entering variables every time. Slack is our main communication tool, so why not play with Chat Ops and trigger the job directly from the Slack window.

Firstly, we needed to configure Slack and Gitlab CI. We created a special Slack channel for this purpose and installed a Gitlab CI app for Slack by clicking “More” and then “Add apps” in the Channel Details screen.

In Gitlab settings → Integrations we added the Slack application. We also needed to enter the project alias to call the tests. In our case, it’s “aurena-tech/system-smoke-tests”. More information on how to do that can be found in GitLab documentation.

In short, the GitLab app ChatOps functionality allows the execution of jobs only from the master branch or branch that is set default in the repository settings, and extra job arguments that are passed in the Slack call are packed into the CHAT_INPUT variable. For that we need to form our call this way:

                /gitlab  <project name> <job name> <job arguments>
            

We use <job arguments> to pass the ENVIRONMENT variable to the test job.

That took care of the user command input. After that, we needed something with Gitlab CI to interpret our commands and map them to the jobs.

For that, we created an extra pipeline for triggering smoke tests from the chat, as you can see:

Fragment from .gitlab-ci.yml
                system:smoke:test:chat:
  extends:
    - .job-system-smoke-test  
  only: [chat]
  environment:
    name: $CHAT_INPUT
            

We also had to adapt the job to process the CHAT_INPUT variable.

Fragment from .gitlab-ci.yml
                .job-system-smoke-test:
  stage: system-smoke-tests
  before_script:
    - if [ -n "$CHAT_INPUT" ]; then export ENVIRONMENT=$CHAT_INPUT; fi
  script:
    - echo "Running tests in $ENVIRONMENT environment"
    - npm ci
    - npm run test-smoke
  artifacts:
    reports:
      junit: junit.xml
            

And that was it! Now we could trigger our tests from the Slack channel.

Typing in the Slack channel /gitlab aurena-tech/system-smoke-tests run system:smoke:test:chat development

The app returns a reply into the Slack room with the job output log.

All together

Here is our combined .gitlab-ci.yml file

.gitlab-ci.yml
                stages:
  - system-smoke-tests
 
# smoke test job definition
.job-system-smoke-test:
  stage: system-smoke-tests
  before_script:
    - if [ -n "$CHAT_INPUT" ]; then export ENVIRONMENT=$CHAT_INPUT; fi
  script:
    - echo "Running tests in $ENVIRONMENT environment"
    - npm ci
    - npm run test-smoke
  artifacts:
    reports:
      junit: junit.xml
 
# pipeline that runs smoke tests from Slack trigger
system:smoke:test:chat:
  extends:
    - .job-system-smoke-test  
  only: [chat]
  environment:
    name: $CHAT_INPUT
 
# pipeline that runs smoke tests from multi-project pipeline or from Gitlab UI
system:smoke:test:dynamic:
  extends:
    - .job-system-smoke-test  
  rules:
    # Never trigger for prod
    - if: '$ENVIRONMENT == "productive"'
      when: never
    # For multi project pipelines
    - if: '$CI_PIPELINE_SOURCE == "pipeline" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'
    # For running the job from Gitlab UI ('Run Pipeline' in CI/CD section)
    - if: '$CI_PIPELINE_SOURCE == "web" && $RUN_SYSTEM_SMOKE_TEST == "TRUE"'
            

Conclusion

We are delighted with the results:

  • More inspection and productivity. Every team member, no matter their technical know-how, can now trigger tests as required and see the results in one place.
  • Code Reuse. Our CI/CD pipeline can trigger system or end-to-end tests from its repository by using Gitlab multi-project functionality.

We look forward to more experimentation with Gitlab Chat Ops functionality.

Article by
Tatjana Statescu

Tatjana is QA lead at AURENA Tech. “The only thing better than finding a bug early is preventing it entirely”: With this in mind, Tatjana makes sure that quality is built into all development stages.

More articles

01.07.2021

Team activities: Racing Experience

The whole AURENA team enjoyed a special in-person event at the Red Bull race track.

03.05.2021

Smoke Testing with Jest and GraphQL

One important capability in our QA strategy is to run smoke tests against remote service endpoints, to check that they are healthy. In this article, we share our approach, walking you through some examples.

12.04.2021

Our approach to Gitlab runners

Until recently our engineering team was exclusively using Gitlab shared runners for our CI/CD pipelines. While they did (and partly still do) get the job done, recent outages have made us aware of how critical functioning CI/CD is to us in our day-to-day work.

23.09.2020

Welcome aboard: Meet QA lead Tatjana Statescu

QA is a cornerstone to ship reliably working products that exceed user expectations. Tatjana is going to build up our competence center to bring quality assurance at AURENA Tech to the next level.

27.04.2020

IaC: AWS CDK vs. Terraform

If you are interested in the difference between IaC tools, check out our comparison of AWS CDK and HashiCorp Terraform.

Open positions

AURENA.tech
text
Product Manager with UX background (f/m/x)

Complement our team by managing product evolution from scratch, as well as enhancing existing solutions along the entire user journey.

  • Leoben (partially remote)
  • Fulltime, permanent
  • Starts at € 65,000 p.a.
AURENA.tech
text
Senior Angular Developer (f/m/x)

Develop new features for our leading real-time auction platform and build new apps from scratch.

  • Leoben or fully remote
  • Fulltime, permanent
  • Starts at € 55,300 p.a.
AURENA.tech
text
Senior Mobile App Developer (f/m/x)

Develop cross-platform native apps and progressive web apps from scratch.

  • Leoben or fully remote
  • Fulltime, permanent
  • Starts at € 55,300 p.a.
AURENA.tech
text
Junior Quality Assurance Engineer (f/m/x)

Create and execute test strategies and test plans for our existing and future applications.

  • Leoben or fully remote
  • Fulltime, permanent
  • Starts at € 39,200 p.a.
AURENA.tech
text
Junior DevOps Engineer (f/m/x)

Design infrastructure for new micro-services, mange operations and advance our internal tooling.

  • Leoben or fully remote
  • Fulltime, permanent
  • Starts at € 44,800 p.a.