HHotTea
DashboardHatchwaysPricingDocsSample report

Reviewer packet · Practical assessment reviewer demo

Fix invite acceptance regression

Alex Chen · Senior Backend Developer

submittedcandidate@example.comaudit chain verified · 6/6 events in chain

Provisioned 2026-04-30T18:02:00.000Z · Submitted 2026-04-30T18:31:44.000Z · Active work ~25 min across 1 session

Reviewer logic map

Read the packet before scoring.

Overview, then the code changes and session replay — the core review — then score. The section bar above jumps anywhere.

1
Confirm overview Candidate, assessment, status, audit chain, and evidence counts.
2
Read the code changes Per-file diff of the submitted work.
3
Replay the AI sessions How the candidate directed the agent, turn by turn.
4
Save scores Enter human reviewer scores and notes at the end.
9/10

Suggested score — reviewer aid, not a verdict

9/10

Heuristic signal (keyword-based, see rubric)

3

Candidate prompts

3

Agent responses

2

AI session records

2

Git snapshots

1

Test runs

6

VM recorder events

Anomaly flags

No anomaly flags.

Evidence checks 3/3 passed

Author-defined checks evaluated against the full evidence record at submission.

Expiry check placed before retryable-member branch in diff

matched in evidence

passed
All 4 test cases pass (4/4 in test output)

matched in evidence

passed
Regression coverage added for expired invite retry path

matched in evidence

passed

AI fluency Reviewer only

6

Total turns

3

Assistant turns

8

Tool calls

2

Write/Edit calls

4

Read calls

2

Bash calls

Models used: claude-sonnet-4-5 · parsed 6 JSONL lines across 1 session(s)

Process analysis Reviewer only

Candidate directed the AI with precise, narrow prompts ("find the smallest fix", "what edge cases should I preserve") rather than open-ended delegation, showing awareness of AI over-reach risk. They verified the fix independently by running all four test cases and reviewing the branch-order change before submitting, rather than accepting the AI-suggested output without validation. Redirection quality and verification habit are both strong signals for senior-level independent judgment.

Activity timeline

Chronology reconstructed from the recorded event chain. Active-work time excludes idle gaps — unlike raw wall-clock completion time.

Active work ~25m across 1 session · wall clock 25m6 events · idle gaps excluded from active time
Session 1 · Apr 30, 18:07–18:31 · 25 min · 6 events
  1. 18:07Candidate promptFind the smallest fix for the expired invite regression before I edit.
  2. 18:07Agent responseStart in src/invites.js and compare branch order against the failing expired retry test. The expiry guard sho…
  3. 18:18Candidate promptI moved expiry above retryableError. What edge cases should I preserve?
  4. 18:18Agent responsePreserve revoked=false plus future expiresAt for pending members, reject revoked invites, and reject active m…
  5. 18:28Test run — passed
  6. 18:31Evidence submittedfinal evidence packet

Code changes

git snapshot at submit · 2026-04-30T18:29:03.000Z. Every line below is candidate-submitted evidence; nothing is edited or summarized.

no diff content found
## git status
 M src/invites.js
 M test/invites.test.js

## git diff
-  if (membership?.retryableError) return true;
   if (new Date(invite.expiresAt).getTime() <= now.getTime()) return false;
+  if (membership?.retryableError) return true;
Earlier git snapshots (1)
2026-04-30T18:08:07.000Z · 33 chars
## git status
clean

## git diff

AI session replay

Reconstructed turn by turn from the candidate's recorded sessions — prompts, agent responses, and tool calls in order. Raw exports stay available under “Inspect raw evidence”.

2026-04-30T18:22:11.000Z · session-test.jsonl · 0 turns (raw — not parseable as session JSONL)
Claude JSONL export: candidate asked Claude to inspect invite tests, review the branch-order fix, run npm test, and explain product/infra follow-up questions.
Terminal capture · 2026-04-30T18:21:36.000Z · agent · 164 chars
Terminal transcript0 commands
codex session: inspected invite tests, identified retryableError branch above expiry guard, moved expiry check before retry path, ran npm test, reviewed edge cases.

Post-interview transcript review Reviewer only

No interview transcript artifact stored yet.

Inspect raw evidence

Unprocessed records exactly as captured — the source of truth behind every rendered section above.

Rubric
{
  "score": 87,
  "score10": 9,
  "taskDecomposition": "observable",
  "verification": "observable",
  "aiCollaboration": "observable in recorded AI session evidence",
  "reviewerConfidence": "usable evidence packet"
}
Audit integrity
{
  "verified": true,
  "coveredEvents": 6,
  "totalEvents": 6,
  "lastAuditHash": "sample-tamper-evident-chain"
}
Developer VM
{
  "status": "recording",
  "provider": "fly-machines",
  "appName": "agentic-evidence-vms",
  "machineId": "e82ee40c0d5758",
  "region": "sjc",
  "image": "registry.fly.io/agentic-evidence-vms:candidate-vm-terminal-router",
  "teardownDueAt": "2026-04-30T22:04:13.000Z"
}
Agent session transcripts (1)
2026-04-30T18:21:36.000Z · agent · 164 chars
codex session: inspected invite tests, identified retryableError branch above expiry guard, moved expiry check before retry path, ran npm test, reviewed edge cases.
Claude session JSONL exports (1)
2026-04-30T18:22:11.000Z · session-test.jsonl · 158 chars
Claude JSONL export: candidate asked Claude to inspect invite tests, review the branch-order fix, run npm test, and explain product/infra follow-up questions.
External AI session evidence (0)

No external AI session evidence uploaded.

Git snapshots (2)
2026-04-30T18:08:07.000Z · 33 chars
## git status
clean

## git diff
2026-04-30T18:29:03.000Z · 240 chars
## git status
 M src/invites.js
 M test/invites.test.js

## git diff
-  if (membership?.retryableError) return true;
   if (new Date(invite.expiresAt).getTime() <= now.getTime()) return false;
+  if (membership?.retryableError) return true;
Final evidence
{
  "tests": "npm test -> 4/4 tests passed. HotTea runner -> 4/4 checks passed.",
  "diff": "Moved invite expiry validation before the retryable membership-error branch and added regression coverage for expired retryable invites, revoked invites, valid pending invites, and already-active members.",
  "notes": "Kept the production change small. Verified the valid pending path still passes after changing branch order."
}
Audit trail
[
  {
    "type": "candidate_prompt",
    "createdAt": "2026-04-30T18:07:10.000Z"
  },
  {
    "type": "agent_response",
    "createdAt": "2026-04-30T18:07:12.000Z"
  },
  {
    "type": "candidate_prompt",
    "createdAt": "2026-04-30T18:18:22.000Z"
  },
  {
    "type": "agent_response",
    "createdAt": "2026-04-30T18:18:24.000Z"
  },
  {
    "type": "test_run",
    "createdAt": "2026-04-30T18:28:51.000Z",
    "passed": true
  },
  {
    "type": "evidence_submission",
    "createdAt": "2026-04-30T18:31:44.000Z"
  }
]
Full event log (6)
[
  {
    "type": "candidate_prompt",
    "createdAt": "2026-04-30T18:07:10.000Z",
    "prompt": "Find the smallest fix for the expired invite regression before I edit."
  },
  {
    "type": "agent_response",
    "createdAt": "2026-04-30T18:07:12.000Z",
    "response": "Start in src/invites.js and compare branch order against the failing expired retry test. The expiry guard should run before retryable membership errors."
  },
  {
    "type": "candidate_prompt",
    "createdAt": "2026-04-30T18:18:22.000Z",
    "prompt": "I moved expiry above retryableError. What edge cases should I preserve?"
  },
  {
    "type": "agent_response",
    "createdAt": "2026-04-30T18:18:24.000Z",
    "response": "Preserve revoked=false plus future expiresAt for pending members, reject revoked invites, and reject active members. Run tests after adding coverage."
  },
  {
    "type": "test_run",
    "createdAt": "2026-04-30T18:28:51.000Z",
    "passed": true
  },
  {
    "type": "evidence_submission",
    "createdAt": "2026-04-30T18:31:44.000Z"
  }
]

Save reviewer scores Reviewer only

Score after you have read the overview, process analysis, diff, and evidence detail.

No reviewer scores have been saved yet.

Copy/paste reviewer note

Formatted for an ATS or assessment-platform review field. This is a reviewer aid, not a hiring verdict.

HotTea reviewer note
Candidate: Alex Chen
Assessment: Fix invite acceptance regression
Score: 9/10 (rubric) · Suggested: 9/10 (AI fluency + verification)
Reviewer scores: not yet scored
Evidence captured: 1 AI session transcript(s), 1 Claude JSONL export(s), 0 external AI evidence upload(s), 2 git snapshot(s), 1 test run(s), 6 recorder event(s)
Summary: Kept the production change small. Verified the valid pending path still passes after changing branch order.
Verification observed: observable
AI fluency observed: observable in recorded AI session evidence
Post-interview transcript review: no transcript artifact stored
Report: https://hottea.ai/report/sess_hatchways_sample?token=rpt_sample
Suggested follow-up questions:
1. Walk through the highest-impact change and why that was the right first move.
2. Explain one tradeoff or shortcut from the assessment and what would change in production.
3. Describe how AI was used, what was accepted or rejected, and how the result was verified.

Interview transcript artifacts

0 transcript artifact(s) stored and summarized for reviewer follow-up. They remain reviewer-only and do not change the automated score.