Sequential Docs
Sign in

AI judging › Use it

Session insights

The replay player's Insights tab judges one session on demand: what the user came to do, how it ended and where they struggled. It uses a daily allowance.

View as Markdown 13 KB

On this page

What it is for

The Insights tab tells you what happened in a recording, and where to look, before you press play.

Examples
  • A support request. A customer writes that billing is broken. Open their session. See if they reached the payment form and if an error from your analytics SDK stopped them. Then jump to where they gave up.
  • A session from Watch. A trial user's session is near the top of Watch. Watch from m:ss opens the player at that moment, with the tab already open.
  • A long session. An engineer asks where to start in a long onboarding session. The heat lane shows where the judge placed the struggle.

How it works

  1. Your browser summarises the recording. The player turns the meaningful events inside the recording into a short text, the digest.
  2. The judge answers your rubric. Sequential sends the digest and your project's current rubric questions to the judge in one call.
  3. Sequential builds the tab from the answers. The judge writes no free text.
  4. Sequential saves the judgment for 7 days.
More about each step
  • The digest includes clicks and page changes that the recorder saw and your analytics SDK did not report.
  • Sequential makes every chip, meter, moment and sentence from the answers. It also uses your rubric's wording and the recording's events and pages.
  • If you or a teammate open the session again, the tab shows the saved judgment with no new call. This is true while the session's events and your rubric's questions stay the same.

Watch ranks by a different judgment. Sequential makes that one in the background, from your analytics events and the recording's DevTools error counts. The tab does not see those counts, so the two judgments can disagree about the same session (Judgments).

What the tab shows

Part What it shows
Verdicts Questions with one answer from a list, such as Intent and Outcome. Each shows the judge's answer and its likelihood.
Signals Yes-or-no questions, such as Rage clicks, Hit an error and Worth watching.
Moments Up to three in the default rubric. Click a moment to jump to half a second before it.
Summary A few sentences built from the answers. A sentence that names a moment links to it.
Footer "Judged just now", or "Cached" with its age, such as "Cached 3 h ago", and Re-run.

A ! on the Insights tab means that the judge marked the session Worth watching.

Verdicts, signals and moments in detail
  • The tab shows these parts from top to bottom.
  • A scale, such as Friction, shows as a meter with the level's name, for example Noticeable struggle.
  • A signal shows no chip when the judge thinks it unlikely.
  • The default rubric's moments are Where they struggled, What they came to do and Where they gave up. Each moment points to one event.

Today the chip works like this:

Signal likelihood The chip shows
Below 40% No chip
40% to 70% "unclear"
70% or more The percentage

A missing chip means the judge thinks the signal is unlikely. It does not mean that your rubric lacks the question.

Two "unclear" chips can mean different likelihoods. Reading a verdict explains how to read likelihoods, including "unclear".

The tab leaves out a moment if the judge thinks it unlikely or cannot place it on one event. So a session with no clear struggle has no Where they struggled.

The heat lane
  • When the judgment loads, red marks appear on the timeline where the judge placed the struggle. A mark is stronger where the judge put more weight.
  • The marks stay while you switch tabs.
  • The marks are not buttons. If you click one, the player seeks there, as it does anywhere on the timeline.
  • To land just before the event, use the moments.

How to use it

  1. Open a session's replay. In Sessions → Latest, click ▶ in the Replay column.
  2. If the side panel is closed, click ▤ Events in the header or press E.
  3. Click the Insights tab. The tab starts a new judgment unless a saved one matches.
  4. Wait while the tab says "Judging this session…". The tab stops waiting after 45 seconds.
  5. Click a moment or a linked sentence. The player goes to that point.
  6. Press play.
  7. To send a teammate there, click ⧉ Share or press C. The link opens on the Insights tab at the current time.
  8. To judge again, click Re-run at the bottom of the tab.

If you click Watch from m:ss on a Watch row, the player opens on the Insights tab. Go to step 4.

What it needs

It needs Details
A recording At least 3 meaningful events inside it (Session replay (web), Mobile apps). Events from before the recording started or after it ended are not judged.
Readable events Clicks need element text and errors need messages.
Judging set up on the server It works even if background judging for Watch is off for the project.
Replay access Every role can open replays, including viewers. So anyone who can watch a session can start a judgment (Roles and permissions).

Limits

  • Every new judgment uses the daily allowance, not only Re-run. The default is 500,000 tokens a day per project, set on the server. The app does not show what is left.
  • The daily allowance can run out. Then the tab says "On-demand judging has used this project's daily allowance." Saved judgments still open.
  • If Sequential cannot read the allowance, it does not judge. It spends nothing. The tab shows "Insights failed" and Retry. Try again after about 30 seconds.
  • Re-run has a 10-minute cooldown per session. After a Re-run makes a new judgment, further Re-runs on that session return the saved one for 10 minutes.
  • The tab has no 👍/👎. They are on Watch rows (Feedback and calibration).
  • Long recordings are trimmed. The judge reads at most 255 events. The tab then says "Judged on X of Y events."
  • No DevTools data. Failed requests and errors that only the recorder captured do not reach the judge. See DevTools in replays.
  • The heat lane needs the tab. It appears only after the Insights tab has loaded in this replay. It does not appear if your rubric has no struggle moment or the judge placed no struggle.
What the judge reads
  • Named custom events such as trial_started show if someone got what they came for (Send custom events).
  • Errors that your analytics SDK sends as events reach the judge.
  • Judging set up on the server is the only setting the tab needs.
When there is no judgment

The tab can show these messages. The failures show under "Insights failed".

The tab says What it means and what to do
"Not enough activity to judge — fewer than 3 meaningful events" Too little happened inside the recording.
"Insights aren't set up on this server" Judging is not turned on for your Sequential server. Ask the person who runs your server to turn it on.
"Timed out after 45 s — try again." The tab stopped waiting. Click Retry. If the server finished the judgment, Retry shows it and makes no second call.
"Sequential could not judge this session. Try again." The judge did not return a usable answer. Click Retry.
"Insights failed (Insights can't run right now. Try again shortly.)" Sequential cannot read the daily allowance, so it does not start a new judgment. It spends nothing. Click Retry after about 30 seconds.
"On-demand judging has used this project's daily allowance." This is not a fault. The tab says when the allowance resets, in your local time and as 00:00 UTC. It also says that saved judgments are not affected. There is no Retry. Wait for the reset.
"The session events did not load. Retry them from the Events tab, then re-run." Click Retry on the Events tab. Then go back to Insights, which tries again.
"The recording did not load, so there is nothing to judge." Click Retry on the player. The tab tries again when the recording loads.
"Insights run once the recording is ready (by 14:05).", with the time when the recording is ready Sequential is still processing the recording. Keep the replay open. When the player loads the recording, the tab judges it with no action from you. If the player stopped checking, select Check now.
"This recording has expired, so it can't be judged." The replay period ended. There is nothing to judge.
The allowance and Re-run
  • During the Re-run cooldown, the tab says "Already judged — same events, same questions, so the verdict below is unchanged."
  • On-demand judging has its own allowance per project per UTC day. It is separate from the allowances of background judging and of Ask. It is also separate from the rubric preview's allowance.
  • The footer shows what each judgment used.
  • If a verdict is on screen when a Re-run hits the limit, the verdict stays. A note at the top of the tab says when the allowance resets. Re-run is hidden until then.
  • A Re-run asks the same questions about the same events. It can move the answers only slightly. It adds no new information.
  • If the events or your rubric's questions change, the tab judges the session again when you open it.
  • If Sequential cannot read the allowance, a Re-run gets "Insights failed", also when a saved judgment exists. The verdict on screen goes away. Retry does not re-run. So it shows the saved judgment if Sequential can read it.
  • Sequential keeps saved judgments in the same store as the allowance. When that store is down, a saved judgment usually does not open either.
How long recordings are trimmed

The judge reads at most 255 events, within a size limit. Sequential keeps these events first:

  • errors
  • rage clicks and dead clicks
  • U-turns
  • form submits
What the digest sends to TypeSafe

For each event, the digest sends:

  • the event's kind and time
  • the page path
  • a label: the element text, form name, error message or event name
  • a short detail

For the session, it sends the length, device type, browser, operating system and country.

Other rules:

  • Sequential removes query strings. It keeps the domain for the entry page, the referrer and page views.
  • The detail of a custom event or an error is its first three properties that are not the SDK's own ($…, [Amplitude] …), as name=value.
  • Sequential cuts each label and detail to 80 characters.

See AI and third parties.