Skip to content

How to create an Acceptance test

How to acceptance test

In this how to guide you will learn how to create, deploy, and run an acceptance test for a Platform service (Frontend Web App or an API) for your team.

Prerequisites

Before adding acceptance tests for your service, you will need to ensure that:

Overview

By completing this guide, you will have completed these actions:

  • Learned how to add an acceptance test for your service.
  • Learned how to run an acceptance test locally.
  • How to customize your pipeline to run an acceptace tests based on tags for different env.

Guide

These tests may include unit, integration, acceptance, performance, accessibilty etc as long as they are defined for the service.

Note

The pipeline will check for the existence of the file test\acceptance\docker-compose.yaml to determine if acceptance tests have been defined.

How to add acceptance test for your service?

You may add tags to features and scenarios. There are no restrictions on the name of the tag. Recommended tags include the following: @sanity, @smoke, @regression refer

If custom tags are defined, then the pipeline should be customized to run those tests as detailed in following sections.

How to run acceptance test locally?

Set required tags, default is empty string which will run all tests

  • pwsh : $ENV:TEST_TAGS = "@sanity or @smoke"
  • shell: export TEST_TAGS = "@sanity or @smoke"

Run the acceptance test script under scripts folder within the repo

1
2
3
docker-compose up -d
cd test/acceptance
docker-compose run --rm wdio-cucumber

How to customize your pipeline to run acceptace tests?

Note

Every pipeline run includes steps to run various post deployment tests. These tests may include unit, integration, acceptance, performance, accessibilty etc as long as they are defined for the service.

You can customize the tags and environments where you would like to run specific features or scenarios of acceptance test

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
postDeployTest:      
  testEnvs:
    performanceTests: snd4, pre1
    accessibilityTests: snd4, tst1,     
    acceptanceTests:
      - env: snd4
        tags: '@demotag'
      - env: dev1
        tags: '@sanity or @smoke'
  envToTest: snd4,dev1,tst1,pre1

If not defined, the pipeline will run with following default settings.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
postDeployTest:      
  testEnvs:
    performanceTests: snd4, pre1
    accessibilityTests: snd4, dev1, tst1    
    acceptanceTests:
      - env: snd4
        tags: '@sanity or @smoke'
      - env: dev1
        tags: '@smoke'
      - env: tst1
        tags: '@smoke or @regression'         
  envToTest: snd4,dev1,tst1,pre1

Please refer ffc-demo-web pipeline:

Test execution reports will be available via Azure DevOps Pipelines user interface for your project and service.

How to disable test?

if you want to disable the test for any reason please refer Disable Test