AI judgments of sessions

For example, judging can answer:

• Did the trial user who opened billing add a card, or give up?
• Did an error stop someone finishing checkout, or were they unsure what to do?
• Which of yesterday's sessions should a product manager watch first?

The default rubric asks:

• What did the user come to do?
• How did it end?
• How much friction did they hit?
• Did an error stop them?
• Does it look like a product bug?
• Is the session worth watching?

It also has a few more signals and moments.

The table shows what Sequential stores for each group in your rubric.

The judge writes no prose. Sequential builds the sentences on Watch from these stored numbers and your rubric's wording. The replay's Insights tab builds its sentences from that tab's own judgment. See Reading a verdict.

Each event becomes one numbered line, except System events. The digest also has a line for each pause of 10 seconds or more.

Each line has the event's time. For anything but a page view, it also has the page the event happened on.

Sequential cuts each name, path, element text or error message to 60 characters.

Sequential cuts each page URL to its path. In other text, Sequential removes the host and query string from a full URL.

Sequential drops a custom-event property if its name starts with $. It also drops a property if its name contains a word such as email, name, phone, address, user, account, ip, card, password or token.

Session facts are:

• duration
• device type
• identified or anonymous
• entry page
• browser
• operating system
• country
• pages visited

If the recording's DevTools capture measured them, the facts also include counts of JavaScript errors, failed requests and console errors.

Some answers are unclear because the digest does not have enough information. Sequential keeps an unclear answer and a "no" apart on purpose.

These views hide what the judge rated unlikely. They do not label it:

• signal chips on Watch
• signal chips in the Insights tab
• weak matches in Ask

See Reading a verdict.

Sequential leaves out what it did not measure. It never replaces it with zero.

A session recorded without DevTools capture has no error counts. The default rubric tells the judge that a missing count means "not measured". A zero would say "nothing went wrong" and pull the answer the wrong way.

The judge does not read weights, question labels, or the label of an option that has criteria. So these changes apply at once.

Watch puts judged sessions into three groups. In each group, the session with the highest Watch score comes first. Watch tells what each group holds.

To calculate the Watch score:

• Each weighted signal adds its weight times the probability of yes.
• Each weighted scale adds its weight times the session's position on the scale.

The default rubric weights Worth watching 45%, Friction 25%, Hit an error 15% and Product bug 15%. Your weights are in the Watch score % field on each signal and scale, under Settings → Projects → your project → Insights.

Sequential calculates the score from the stored answers and your current rubric each time you open Watch. So a weight change never needs a new judgment.

Ranking follows your current rubric at once. A removed or turned-off question stops counting toward every session's Watch score. An older judgment scores nothing on a question added after it, so it can rank lower. See Watch.

The replay's Insights tab judges one recording on demand. It builds its own digest in your browser, from the recording and its events. It differs from the background digest:

• It adds clicks and page changes from the recording.
• It leaves out the DevTools counts.
• It keeps the host, such as app.example.com, on page views and the referrer.
• It sends up to 3 properties of each custom event or error, without the name filter.
• It does not remove hosts or query strings from text such as an error message or a property that holds a link.

When you open the tab, it judges the recording with your current rubric. It reuses the result for up to 7 days.

Re-run forces a new judgment, at most once every 10 minutes per session. On-demand judging has its own daily allowance per project.

The tab can judge a recording while background judging is off. It never changes what Watch uses, so the two can disagree. See Session insights.

The judge knows only what your events tell it.

• Name your milestones. Custom events such as trial_started or invite_sent show whether someone got what they came for. Put the meaning in the name, because the judge reads only the first 80 characters of the properties. See Send custom events.
• Make events readable. A click with no text or id reads as "element". A screen with no path or name reads as "(unnamed screen)". An error needs its message.
• Record with DevTools capture. A session can break without sending an error event. Then the DevTools counts of errors and failed requests can be the only sign. See DevTools in replays.
• Ask answerable questions. The judge always gives an answer. A guess looks the same as a measurement. Ask whether they opened the docs, not whether they read them carefully.

The judge reads at most 255 events. Sequential gives priority to errors, rage clicks, dead clicks, form submits and page views. If it must drop events, it drops custom events and plain clicks first.

If the text is still too large for the judge, Sequential retries once with the higher-priority half of the events. That judgment ranks like any other. On hover, a Watch row's expanded details show "Based on X of Y events".