Running tests and reviewing reports

Back to the guide index

Run a suite

1. Select Run in the header.
2. Select a suite from Run Target.
3. Confirm the Active connection is the intended environment.
4. If the deployment presents an execution selector, choose Local, Background, Docker, or Browser. The available choices depend on deployment configuration.
5. For Local, Background, or Docker execution, confirm that the environment is connected. For Browser, confirm that the environment has a D365 URL and that the Testgrity Recorder extension is installed and detected.
6. Select Run suite.

The portal starts the suite with the selected runner, queuing it when the deployment uses execution workers. Only one suite per user can be queued or running at a time. If the saved D365 session cannot satisfy MFA, a local headed runner may leave its browser open for manual sign-in.

Run in Browser

Browser sends the suite to the Testgrity Recorder extension and runs it sequentially in the current Chrome or Edge profile. This mode uses the D365 session in your browser instead of Testgrity's saved automation session, making it useful for MFA, interactive debugging, and environments that cannot use stored automation credentials.

When the run starts, Testgrity opens a D365 tab. Keep that tab open until every case finishes. The extension displays the final passed or failed result, starts a 15-second countdown, and then closes the test tab automatically.

Browser execution currently supports common recorded actions, including opening apps and navigation targets; opening records, tabs, command buttons, and Save; setting and clearing fields; keyword filtering; key presses; waits; and text or field-value checks. If a case contains an unsupported advanced action, that step fails with a message identifying the unsupported action. It does not fall back to another runner.

Browser runs do not currently generate Playwright traces or videos. Background execution remains the recommended mode for unattended or parallel runs.

Interactive connections can also run in the background while their encrypted MFA-authenticated session remains valid. If Microsoft redirects a worker to sign-in, MFA, or Conditional Access, Testgrity stops the run, marks the connection as expired, and asks you to reconnect it in the browser before trying again.

Monitor or cancel a run

Suite and case statuses update through these states:

• Queued — waiting for a worker.
• Running — the automation runner is executing the test.
• Passed — the observed outcomes matched the expected outcomes.
• Failed — at least one result did not match expectations or execution failed.
• Cancelled — cancellation was requested before completion.

The suite log shows the environment, execution method, total, active, successful, and failed cases plus the success rate. The method is Background, Browser, Docker, or Local; runs created before method tracking was added show Unknown. Each suite log displays a readable monospace run ID in the top-right area, such as RUN-20260719-0029. The date-based, workspace-scoped sequence makes a run easy to reference when discussing results or troubleshooting, while Testgrity retains its internal identifier separately. Select Cancel on an active suite log to request cancellation. An action already in progress may take a short time to stop.

Expand Test Case Reports while a case is running to see its live step progress. The case card shows the number of completed steps, the total number of steps, and a percentage. Progress updates as each step finishes; the Report button becomes available after the case reaches a final status.

Suite logs are shown 10 per page. Use the previous and next buttons to move through older runs.

Configure run evidence

Open the profile menu and select Settings to choose which evidence is created for future runs.

• Screenshot capture turns screenshots on or off.
• Screenshot mode controls whether screenshots are captured for every step, failed steps only, the final result only, or not at all.
• Retain screenshots for keeps screenshot files for 1, 7, 14, or 30 days.
• Video recording turns run videos on or off.
• Retain videos for keeps video files for 1, 7, 14, or 30 days.
• Generate trace creates a downloadable diagnostic trace for each test case run.

Saving a shorter retention period removes expired screenshot and video files from the workspace. These settings apply to future runs. Existing reports keep their recorded step status and error details even when an evidence file is not available.

Open a case report

1. Expand Test Case Reports beneath a suite log.
2. Expand a case for its summary, expected result, actual result, and error message.
3. Select Report for the full run report.

Read the full report

The report summarises the run's environment, positive or negative test type, actual status, duration, and step counts. The Step log contains each action's duration, expected outcome, target, value, result, and error details.

Available evidence may include:

• A screenshot for an individual step.
• A run video.
• A diagnostic trace for timeline, network, console, and page inspection.
• Detailed HTML and JSON execution reports.

Artifact buttons are shown only when that file exists for the run. Availability depends on the workspace settings, retention policy, deployment configuration, and the runner that executed the test. Use Back to portal to return to the same suite and run context.

Interpret positive and negative tests

• A Positive test expects every step to succeed.
• A Negative test contains at least one step expected to fail.
• The report passes when observed step outcomes match those expectations.

For example, a validation step marked as expected failure is successful when the application rejects the invalid operation. The report distinguishes the expected failure from an unexpected automation error.