Access regression testing (preview)

Check access in a normal case

The normal designer now includes Check access under Validation & assertions. Create new access tests as normal cases. Saved access cases remain editable in the existing editor; the New access case creation option has been removed.

1. Choose New → New case, or open an existing case.
2. Add Check access from the toolkit. Select the case's Execution persona above the steps; every access step uses that same persona.
3. Choose Entity permissions or Record access, then select the entity. Entity display names appear first; logical names are shown underneath.
4. Add only the access rules needed for this test.
   • Entity permissions compare an exact level: No access, User / team level, Business unit, Business unit and child units, or Entire organization. Supported actions include Create in this mode.
   • Record access compares Allowed or Denied for each selected action. Choose the current record opened by earlier steps, a named record created by an earlier Dataverse step, or an existing test record GUID. Create is available only in Entity permissions.
5. Save and execute the suite in Background mode from Run. Access steps can appear before or after normal business-process steps in the same case.

Entity permissions use entity privilege metadata and RetrieveUserPrivilegeByPrivilegeId, including the full depth inherited from team roles. The check compares the highest returned depth across business units; verification details retain the business-unit references. User / team level corresponds to Dataverse Basic depth, and does not prove a record is accessible only to its owner. Record sharing, team ownership, and hierarchy access can affect particular records; use Record access for those checks. Unsupported depths such as RecordFilter produce an execution error. Neither mode changes permissions or mutates records.

Each configured rule retains expected and observed access in shared run logs, Reports, and JSON exports. An execution error cannot be treated as an expected successful denial. Access steps require a verified persona and a suite run. Cases containing access checks suppress screenshots, video, and traces; structured access evidence is retained. Historical snapshots remain available after editing or deleting the ordinary case.

Run reports highlight the execution persona and username saved with that result. Suite summaries show the persona for each case, and JSON and CSV exports retain the same identity snapshot. Editing or deleting a persona does not change historical reports. Results without identity metadata show No persona recorded.

Existing access-case editor

Saved access cases remain available in Designer → Suites & Cases with an Access badge. Select one to edit its record-right expectations. This editor requires PostgreSQL and existing test records. New access tests use New case → Check access.

1. Choose a project and connection. Add personas using Manage connections.
2. In Designer, select an existing case with an Access badge. Its suite and connection remain fixed.
3. Enter a case name and select one execution persona. All checks in the case use that persona. Use the normal designer for new cases targeting other personas.
4. Add an entity, describe its scenario and intended access, and supply an existing record GUID. For each right (Read, Update, Delete, Assign, Share, Append, Append to), choose Allow, Deny, or Not tested. Add more entities or record scenarios as needed. Each configured right becomes an independent suite check.
5. Select Save access case. The case appears once in Suites & Cases with an Access badge, persona name, and check count. Cases with all rights set to Not tested remain available for authoring. Ordinary cases open the step editor.
6. Open the Run tab, select the suite, and start Background execution. Access checks run with the other cases in that suite and use the same suite-run logs. Existing schedules and CI suite triggers can also run its checks. Review individual persona results in Run and Reports. Designer only authors cases.

Use Delete in the access editor to remove the case and its generated checks. Historical reports remain available. Editing a case preserves the other scenarios in its suite. Saved cases retain their connection and suite; use New case → Check access when targeting a different connection or suite. The former /access URL redirects to Designer.

Existing matrix scenarios open as separate cases for each configured persona. Their generated check IDs and historical reports are retained when saved in the new editor. Legacy scenarios without expectations remain drafts until a persona is selected.

What the existing editor tests

Every check verifies the authenticated Dataverse user with WhoAmI, resolves the table's entity set, then calls RetrievePrincipalAccess as that persona. The reported result is the effective right on that specific record. Allowed reads also retrieve only the primary key to verify that the record can be read.

Update, Delete, Assign, Share, Append and Append to checks inspect rights; they do not attempt those operations. A denied read is evidence of missing effective ReadAccess, not an attempted read-operation denial. Create, app entry, protected fields, and mutating operation tests are not supported in this preview.

Scenario labels describe records you supply. The preview does not provision fixtures or verify that a record's owner, business unit, team, or sharing matches the label. Configure the records beforehand and keep them available between runs.

Microsoft describes the record-rights API in Verifying access in code.

Existing editor results and evidence

• Passed: effective access matches the reviewed expectation.
• Access mismatch: access is unexpectedly granted or denied.
• Execution error: access could not be established. Login errors, insufficient rights to query effective access, missing records, timeouts, throttling, or invalid responses never count as successful denial tests.

Execution errors map to the existing Failed run status for schedules, exports, and CI. Suite-run logs and case reports identify them separately. Results retain the policy revision, expected/observed access, verified user, scenario, record reference, returned rights and whether a read was verified. Structured evidence is included in the existing JSON report export. Access cases suppress screenshot, video and trace capture to avoid collecting session or record contents.

Case reports compare access changes against up to 500 recent results. Changes compare the same persona, record, scenario, operation, expectation and policy revision. A revised policy starts a new comparison group. Saving creates a new immutable revision for future runs; queued and historical runs retain their original case snapshots.

Edit or remove generated checks through their access case in Designer. Changing a right to Not tested removes the generated case without changing historical reports. Concurrent edits are rejected with a reload message rather than overwriting another author's work.

Optional AI for the existing editor

An administrator can enable Azure AI Foundry assistance for selected workspace IDs. Foundry supports compatible chat deployments from different model providers. The active deployment is selected in server settings; its provider and deployment name appear in the AI panel. Tests remain usable with AI disabled.

Select a persona first. Draft with AI, above the entity scenarios, sends the entered requirements, table names, and selected persona label to Azure AI Foundry. Review its suggestions and choose Add reviewed entities. The editor groups suggested rights by entity and scenario. Supply record IDs, review each right, then save the policy. Suggestions never run automatically. The policy retains the AI request reference and the server records provider, model/deployment, prompt version, and a source fingerprint.

Explain result in the case report sends a small structured evidence summary. It excludes record IDs, usernames, scenario text, business field values, credentials, screenshots, traces, and raw error messages. The response is advisory and cannot change the recorded outcome or permissions.

AI requests have a 45-second timeout, a 3,500-output-token limit, and a shared workspace limit of 20 requests per hour, including failed attempts. Setup is documented in the repository README and Azure Web App settings guide.