Suite uses Cypress to run e2e tests.
Trezor-user-env is a docker image which provides all the necessary instrumentation required to run test (bridge and emulators).
The run_tests.js script is the entrypoint for e2e tests. It:
- picks tests files to be run (see @tags)
- retries tests if needed (see @retry)
- reports tests results
Each test should be assigned a tag at the top of the test file. These allow you to add more fine grained control in run_tests.js.
At the moment, there are following tags:
Assigning a @group allows run_tests.js script to sort the test files into groups and run them in parallel on CI. At the moment these groups exist:
If there is a test that you for any reason need to retry if it fails you may provide
@retry=2 tag. In this
case, test will be ran 3 times in total and count as failed only if all runs fail.
There is a tool to track tests runs and their results, temporarily hosted here https://track-suite.herokuapp.com/ Repo here: https://github.com/mroz22/track-suite
Suite test opens cypress test runner and prepares everything to run tests.
Disclaimer: GUI from docker container does not work on mac. Only linux is supported at the moment.
It is possible to run tests with image snapshots to test for visual regressions. To enable snapshots, use env variable:
When you need to update image snapshots you have 2 options:
- use CI job. This will generate new snapshots in artifacts together with a handy script that updates your snapshots locally. Check the log output.
./docker/docker-suite-snapshots.sh. This does the same as
./docker/docker-suite-test.sh, the only difference is it won't fail on non-matching snapshots but generate new snapshots.
Cypress has 2 problems when communicating with trezord.
- Does not send request origin headers correctly -> problem with CORS.
- Tests in CI fail occasionally on POST to '/' with 403 -> problem with CSFR.
- Change default address to 0.0.0.0 instead of 127.0.0.1 to make it work on mac without any other configuration
As a workaround we have a custom binary compiled from this trezord branch