Running tests and reviewing reports

Back to the guide index

Run a suite

1. Select Run in the header.
2. Select a suite from Run Target.
3. Confirm the Active connection is the intended environment.
4. Choose Background or Browser from the execution selector.
5. For Background execution, confirm that the environment is connected. For Browser, confirm that the environment has a D365 URL and that the Testgrity Recorder extension is installed and detected.
6. Select Run suite.

The portal starts the suite with the selected runner, queuing Background runs for an execution worker. Only one suite per user can be queued or running at a time.

Run in Browser

Browser sends the suite to the Testgrity Recorder extension and runs it sequentially in the current Chrome or Edge profile. This mode uses the D365 session in your browser instead of Testgrity's saved automation session, making it useful for MFA, interactive debugging, and environments that cannot use stored automation credentials.

When the run starts, Testgrity opens a D365 tab. Keep that tab open until every case finishes. The extension displays the final passed or failed result, starts a 7-second countdown, and then closes the test tab automatically.

When video recording is enabled, the D365 tab waits before the first step. Select the Testgrity extension button once to authorize tab capture and start the run. Chrome and Edge require this explicit user action. The extension records a separate WebM video for each test case.

Browser execution currently supports common recorded actions, including opening apps and navigation targets; opening records, tabs, command buttons, and Save; setting and clearing fields; keyword filtering; key presses; waits; and text or field-value checks. If a case contains an unsupported advanced action, that step fails with a message identifying the unsupported action. It does not fall back to another runner.

Browser runs create screenshots and videos according to workspace settings. When trace generation is enabled, Browser mode creates a Testgrity browser-diagnostics ZIP containing the step timeline, page URLs, results, errors, and artifact links. It is intentionally distinct from a Playwright trace because the extension does not run inside a Playwright browser context. Browser results also receive HTML and JSON summaries generated by the Playwright reporters after the extension finishes each case. Background execution remains the recommended mode for unattended runs and genuine Playwright traces. Portal runs are sequential by default; controlled concurrency is available only to deployments or API integrations configured to use it.

Interactive connections can also run in the background while their encrypted MFA-authenticated session remains valid. If Microsoft redirects a worker to sign-in, MFA, or Conditional Access, Testgrity stops the run, marks the connection as expired, and asks you to reconnect it in the browser before trying again.

Monitor or cancel a run

Suite and case statuses update through these states:

• Queued — waiting for a worker.
• Running — the automation runner is executing the test.
• Passed — the observed outcomes matched the expected outcomes.
• Failed — at least one result did not match expectations or execution failed.
• Cancelled — cancellation was requested before completion.

The suite log shows the environment, execution method, total, active, successful, and failed cases plus the success rate. The method is Background or Browser; runs created before method tracking was added show Unknown. Each suite log displays a readable monospace run ID in the top-right area, such as RUN-20260719-0029. The date-based, workspace-scoped sequence makes a run easy to reference when discussing results or troubleshooting, while Testgrity retains its internal identifier separately. Select Cancel on an active suite log to request cancellation. An action already in progress may take a short time to stop.

Expand Test Case Reports while a case is running to see its live step progress. The case card shows the number of completed steps, the total number of steps, and a percentage. Progress updates as each step finishes; the Report button becomes available after the case reaches a final status.

Suite logs are shown 10 per page. Use the previous and next buttons to move through older runs.

Configure run evidence

Open the profile menu and select Settings to choose which evidence is created for future runs.

• Screenshot capture turns screenshots on or off.
• Screenshot mode controls whether screenshots are captured for every step, failed steps only, the final result only, or not at all.
• Retain screenshots for keeps screenshot files for 1, 7, 14, or 30 days.
• Video recording turns run videos on or off.
• Retain videos for keeps video files for 1, 7, 14, or 30 days.
• Generate trace creates a Playwright trace for Background runs and a browser-diagnostics ZIP for Browser runs.
• Retain traces for keeps traces or browser-diagnostics ZIP files for 1, 7, 14, or 30 days.

Saving a shorter retention period immediately removes expired screenshots, videos, traces, and native reports containing expired artifacts from the workspace. Capture settings apply to future runs; retention changes also clean up artifacts that are already expired. Existing reports keep their recorded step status and error details even when an evidence file is not available.

Open a case report

1. Expand Test Case Reports beneath a suite log.
2. Expand a case for its summary, expected result, actual result, and error message.
3. Select Report for the full run report.

Read the full report

The report summarises the run's environment, positive or negative test type, actual status, duration, and step counts. For a failed run, Failure detail shows a concise explanation and lets you expand the underlying technical detail.

The Step log contains each action's duration, expected outcome, target, configured value, actual result, and error details. Steps that work with multiple fields list the configured field names and values. Failed steps provide an Observed failure summary and expandable technical detail when available.

Available evidence may include:

• A screenshot for an individual step.
• A run video.
• A diagnostic trace for timeline, network, console, and page inspection.
• Detailed HTML and JSON execution reports.

Screenshot, video, and trace links are shown when their files are available for the run. Native Playwright HTML and Native Playwright JSON remain visible but disabled when those reports are unavailable. Availability depends on workspace settings, retention policy, deployment configuration, and the runner that executed the test. Use Back to return to the same suite and run context.

Interpret positive and negative tests

• A Positive test expects every step to succeed.
• A Negative test contains at least one step expected to fail.
• The report passes when observed step outcomes match those expectations.

For example, a validation step marked as expected failure is successful when the application rejects the invalid operation. The report distinguishes the expected failure from an unexpected automation error.