# Hatchways Reviewer Calibration Packet

Give Hatchways reviewers a concrete rubric for reading AI-assisted assessment evidence without turning the packet into an automated hiring verdict.

## Sample Baseline

- Assessment: Fix invite acceptance regression
- Reviewer aid score: 9/10
- Hidden tests: 3/3 passed
- AI fluency: parsed 6 JSONL lines across 1 session(s); models: claude-sonnet-4-5; tool calls: 8
- AI session transcripts: 1
- Claude JSONL exports: 1
- Git snapshots: 2
- Test runs: 1
- VM events: 6

## Calibration Dimensions

### Problem decomposition

Question: Did the candidate identify the failing behavior, relevant files, and test boundary before changing code?

Evidence to inspect:
- assignment packet access
- first transcript turns
- terminal commands
- git snapshot before edit

Strong signal: Candidate inspected README/tests/logs, named the invite acceptance path, then made a focused change.

Follow-up signal: Candidate jumped straight to implementation or relied on AI output without establishing the failing path.

### AI collaboration

Question: Did the candidate direct Claude/Codex with useful context and evaluate the output?

Evidence to inspect:
- Claude JSONL exports
- Codex/Claude transcript count
- tool call summaries
- AI process analysis

Strong signal: Candidate used the agent to compare hypotheses, inspect files, and review a patch rather than outsourcing the whole answer.

Follow-up signal: Transcript shows one-shot generation, no verification prompts, or final code that has no visible reasoning trail.

### Verification discipline

Question: Did the candidate run the right tests and connect the result to the final claim?

Evidence to inspect:
- public test output
- hidden-test summary
- final evidence notes
- GitHub Action event

Strong signal: Public tests and hidden checks pass, with the candidate explaining what changed and what remained untested.

Follow-up signal: Passing status appears only after unexplained code changes, or the final note overclaims coverage.

### Engineering judgment

Question: Was the final patch narrow, maintainable, and aligned with the task constraints?

Evidence to inspect:
- diff summary
- changed files
- reviewer note
- anomaly flags

Strong signal: Patch is small, locally justified, and does not introduce unrelated refactors or hidden state.

Follow-up signal: Large paste bursts, broad rewrites, missing rationale, or transcript/diff mismatch need reviewer attention.

### Communication

Question: Can the reviewer understand the candidate's tradeoffs and confidence without a live debrief?

Evidence to inspect:
- candidate final notes
- ATS-ready reviewer note
- follow-up questions

Strong signal: Candidate names the failure mode, fix, tests run, and residual risks in plain language.

Follow-up signal: Candidate leaves reviewers to infer intent from code and telemetry alone.

## Decision Bands

- 8-10: Strong evidence packet. Reviewer can focus on code quality and team fit rather than reconstructing the work trail.
- 5-7: Usable packet with follow-up questions. Candidate may have solved the task, but process evidence or verification is incomplete.
- 1-4: Weak packet. Passing tests alone should not carry the review without a live follow-up or deeper code inspection.

## Reviewer Prompts

- Which claim in the final note is directly supported by transcript, terminal, git, or test evidence?
- Where did the candidate catch or correct AI output?
- What would Hatchways hidden grading see that the candidate could not have known?
- Does the final patch match the path the candidate described?

## Not Claimed

- The score is a reviewer aid, not an automated hiring decision.
- The packet standardizes evidence; it does not claim perfect outside-AI prevention.
- Hatchways hidden tests and human reviewer judgment remain the system of record.

## Proof URLs

- Calibration JSON: https://hottea.ai/hatchways/calibration.json
- Sample reviewer packet: https://hottea.ai/sample-report
- Sample reviewer Markdown: https://hottea.ai/sample-report.md
- Buyer packet: https://hottea.ai/hatchways/packet.md
- Integration guide: https://hottea.ai/hatchways/integration.md
- Pilot kit: https://hottea.ai/hatchways/pilot.md