Events

• yath uses Test2 events directly, serialized as json.
• No Test2 -> TAP -> Test2 conversion (lossless!)
• Any/All meta-data and debug info (file, line, etc) is preserved and available
• stderr/stdout are converted to events
• TAP parsing still occurs for tests that opt-out, or just print TAP

Project config

.yath.rc is intended to be a project config that specifies plugins, preloads, and default arguments. This SHOULD be commited to the repository.

.yath.user.rc is intended to be local overrides to .yath.rc, it SHOULD NOT be commited to the repository.

Both files have the same layout:

Event Timeout

You can configure the timeout either at the command line, or provide test-specific timeouts via directives written in the test itself:

Post-Exit Timeout

This is also configurable:

Overview

In-Test Directives

Directives are special comments at the top of test files that tell the harness the test needs special treatment.

There are 2 forms for directives, the first type is used to set key/value attributes:

The other style is for toggling behaviors on or off:

Categories

Categories

This is the default category, implies nothing special.

Tests with this set cannot run concurrently with any other tests that set this.

This test cannot run with any other tests, it must be run non-concurrently.

This test takes a long time to complete. In a concurrent run this will be run as soon as possible.

Currently an alias to 'long' but may not always be.

Config

Configuration

Set the stage to run in (only useful with custom preloads).

Set the event timeout in seconds.

Set the post-exit timeout in seconds.

Set a custom key/value pair.

Do not run this test with preloads, start a clean process to run it. Currently this is identical to NO-FORK but that may not always be the case.

Do not run tests in a forked process, start a new process instead (fork+exec).

Wait forever, do not timeout.

Do not use the Test2 Stream formatter, use TAP instead.

Temporary Directories

• Yath has a single temp dir under which all temp data is kept
• When requested, yath will put the temp dir into shm
• Every test that yath runs has a temporary directory (under the main one) where output and meta-data are stored.
• Each test has the $TMPDIR env var set to a test-specific temp dir under the tests dir.
• This makes cleaning temp files easy, everything is nested under the main temp dir.

Email

Email Notifications

Email the test results (and any log file) to the specified email address(es)

Email the owner of broken tests files upon failure. Owner is specified by a test directive.

Usually owner failures are sent as a single batch at the end of testing. Toggle this to send failures as they happen.

Slack

Slack Notifications

You can have the results sent to a slack channel:

You can also just have the failures sent to a slack channel:

Optionally you can have the log file attached: