Review a task

Describe the task

On Tasks, choose Describe a task. Say what someone needs to get done in your words ("invite a teammate with the right access"), where they start, and, if you like, why it matters: who depends on it, or what goes wrong when it fails. The assessment uses why it matters to judge which problems matter most; the attempts never see it.

Say how you'll know it worked

In your own words, say how someone would know the task worked, like: the new teammate shows up as pending. Put what appears on screen in quotes, and Suggest checks turns your words into checks for you to confirm or change. A check is a rule the browser applies itself when the attempt stops, so “seen” is a fact: words appear on screen, an item appears in a list, a value stays after reloading, the attempt ends on a page, words go away, an email arrives (read from your staging site’s captured email), or a file downloads (Variantly keeps only its name). A task can have up to three checks, and all must be seen. Under each, Checks says what it reads and Doesn’t check says what it can’t establish.

Attempts never see how you'll know it worked. So when a check looks for something an attempt has to type, like the teammate it invites, give it under Details to use (“Invite ana@yourcompany.com as a member”). The form and the task's readiness say when a check names something the task doesn't give. An attempt writes its own words where a person would, but it never makes up a value the task depends on: without one, it stops and says what's missing.

When no check fits, let the assessment judge it: it reads the attempt’s screens against your words and says whether the task was done. The result says judged done or judged not done, never seen.

Choose who is trying

Add one or more perspectives, such as a new administrator, each with a few traits: their role, how well they know the product, what they know, and what limits them. Each AI attempt uses one perspective, so you can compare where those attempts stop or take a different path. A perspective can name the test account it signs in with, like admin or member, so it sees what that account can. Each trait is tagged with where it came from: "from" a file, "inferred from" a file, or "assumed". A trait you type is assumed. A perspective shapes how an attempt explores. It isn't evidence that people behave that way.

Add your files

Files are optional. Upload research notes, requirements, or support and analytics exports as Markdown, text, CSV, JSON or PDF (text only), up to 10 MB each, with 25 files in use per product. Add the file's date if you know it. Otherwise it shows as "Date unknown". Variantly reads the files without following links or running anything in them, and draws statements from them: what the product is meant to do, who uses it, and what research, support and analytics report. "From a file" means the file says it; "inferred from a file" means a model read it that way from what the file says. Neither has been checked against your product.

Variantly suggests tasks from the statements. Each suggestion cites the file and the part it rests on, with its counts and dates. Accept, edit or dismiss each one. A suggestion has no expected result, so you add one when you accept it. Dismissing asks why.

Choose what a review reads

A task's page lists, under Context for this task, the statements the next review's assessment reads, and why each is there: you pinned it, a perspective cites it, the suggestion the task came from cites it, or it shares words with the task. Pin a statement so every review of the task reads it, up to 20, or leave one out so none does. A file you upload on the task's page is pinned to it once it's read. The attempts themselves never see the statements: they get only each perspective's traits.

A review keeps the statements it was given. Its page lists them under What the assessment read, and a finding names the ones it relied on.

Removing a file stops new reviews, suggestions and pins from using it, and Add back undoes that. Reviews already run keep what they used. Deleting a file removes the original, its statements, its pins and the suggestions that cite it. A task made from a suggestion stays, without the citation, and earlier reviews keep a record that a statement was read, but not its text.

Choose where attempts run

On Tasks, open Where attempts run. There are three places, and the page marks the one in use:

• Look-only on your live site, with nothing to set up. Attempts read and navigate your public pages, sign in to nothing and submit nothing. They show what the AI could reach by navigating public pages. Completing a task may require sign-in or form submission.
• On your live product. Choose Public pages, no sign-in for a task that needs no account. Each attempt starts in a fresh browser with no account session, and its perspectives must have no account role. For account access, list the accounts attempts sign in with, each by a role like admin or member, and agree that attempts may submit forms and change data there. Attempts then do the task for real, like sending a chat message or adding an invoice line, and can write their own text and upload a sample file. Variantly checks every request the browser sends and stops the attempt before it pays or enters card or bank details, emails, invites or messages anyone outside your company, or changes passwords, billing, API keys or member roles. Deleting, canceling and signing out are refused too, unless you allow them. What attempts create stays in your product, and they run one after another.
• On a staging site: a copy of your product that isn't live, like staging, QA, a sandbox or a preview, for separate data or flows you'd rather not run on your live product. Enter its address, choose public-page or account access, and agree, as for your live product; the same limits apply. A reset address, if you have one, gives each attempt fresh accounts, so attempts run side by side. When a staging site is set up, attempts use it. How to set up a staging site.

Scope's test account signs in to your live product to scan it for the report, and changes nothing. It can also be an account attempts sign in with. Passwords are sealed when you save and never shown again. Variantly checks what the browser sends; only you can vouch for what your server does with it.

Check readiness and run a review

Readiness names the kind of review the task can run in one sentence, and lists what a full task rests on, each with a word. "Configured" means Variantly holds it. "Owner confirmed" means you agreed to it, and the row shows your name and the date.

• Full task: attempts start as public visitors or sign in with the configured accounts, submit forms and check the result, inside the fixed limits, on your live product or your staging site.
• Look-only: attempts read and navigate but don't submit, so the review can't show whether the task completes. Let attempts submit to see that.
• Can't run yet: the sentence says why, with one action.

Choose 1 to 6 attempts for each perspective, 12 at most in all, then choose Run review. More attempts show whether a result repeats. They don't show how many people would hit it.

Read the result

The review page shows each attempt as it runs, then becomes the result. A review is marked "No model" or "With a model". Its first lines count each kind of result apart, and never add them together:

• The expected result check, in attempts: for example "Seen in 2 of 4 attempts, couldn't tell in 2." Each attempt gets one word: Seen, Not seen, Couldn't tell or Not checked. An attempt that stopped before the check is Not checked.
• Usability: its findings and how many need a decision, or "Not assessed, no model ran."
• Accessibility rule checks: findings and the screens they are on, or "No findings" with the number of screens checked.

When no finding waits for a decision, Next says why in one sentence. If the check couldn't tell or didn't see the result, it says in how many attempts, and which page the check read when Variantly recorded it. Its main action opens the steps and screens of the first such attempt. Edit the expected result sits beside it. Otherwise Next offers Run the review again.

The Attempts table shows how each attempt ended and what its check found. Each attempt opens to its steps, a screenshot of each screen, and the screen its check read.

Accessibility rules run on every screen the attempts captured. Those findings stay with the review: they are not rows of your report. Usability findings are proposals from reviewing the attempts, each with the screens it rests on, why it might matter, a change to consider and a question to check with people. A review always says what it didn't cover.

Decide

Open a finding to see its evidence beside the decision. Choose Act on it, Check with people, or Intended behavior, and say why. Act on it unlocks once the evidence is open. Each decision is signed with your name and the date, and earlier ones stay. An intended-behavior decision never hides an accessibility finding on the same screen.

Recheck after a change

After you act on a finding, Recheck this runs the same task again from the same perspective. It shows the original and fresh evidence side by side, the condition, and a result:

• Seen: the condition appeared again on the same screen.
• Not seen: the fresh attempt reached the same screen, the check ran there and found nothing, and what Variantly compares matched. This doesn't show the problem is gone.
• Couldn't tell: the fresh attempt stopped, wasn't assessed, or differed from the original. The reason is shown.
• Not checked: the fresh attempt took another path and never reached the screen.

Under the result, a comparison lists what matched and what differed: perspective, account role, environment revision, start page, screen size and build. Anything Variantly couldn't compare is named as unknown.