Talent Dril
What you get back.
Built to be forwarded internally without editing. Every score cites a timestamp. Read the tape and decide for yourself.
Candidate A
ExampleAI Architect, LLM Systems
Verification checklist
Every mark earned live, in the exchange itself.
War story held under P1 and P2 with specific, lived detail.
Named a real tradeoff and the failure that prompted it.
Demonstrated depth: quantized one model family, not the full inference stack
Real work on one model family; the claimed stack-wide 60% cost cut was not demonstrated.
Could not describe a real decision they owned in this dril.
Requirement match
Measured against what the candidate demonstrated in the room.
Must-have
Preferred
Evaluation
Rests on 5 of 5 dimensions
Five dimensions, not one pass or fail number.
Survived P2, added specific detail on the fallback chain.
Correct with lived detail; named the regression the eval framework caught.
Claims 'I led' but the indexing decision stayed someone else's.
Named the real scope gap and reasoned from first principles instead of bluffing.
Answer-first, followable on the first pass.
What stood out
Strongest
“The RAG eval pipeline flagged silent regressions before they ever hit production. That's the whole reason I built it that way.”
07:20
Weakest
“I led that migration, mostly.”
09:45
Most revealing
“Honestly the quantization only covered our largest model family. I didn't get to the rest before I left.”
11:10
Named the real scope plainly instead of letting the 60% figure stand unqualified.
Flags
Interview video & transcript
Interview video (example)
On a real Job Dril Card, this shows the candidate's full interview video.
- Tell me about a production system you've built end to end.02:10
- “I built out our multi-agent LLM orchestration layer. It handles routing between four different model backends depending on task complexity.”02:25
- Walk me through what happens when one of those backends goes down mid-request.02:50
- “We have a fallback chain. If the primary model times out it retries against a secondary, then a cached response if both fail.”03:10
- Tell me about the RAG evaluation work.06:50
- “The RAG eval pipeline flagged silent regressions before they ever hit production. That's the whole reason I built it that way.”07:20
- You mentioned leading the vector search migration. Walk me through a decision that was yours alone.09:20
- “I led that migration, mostly.”09:45
- What part wasn't yours?09:58
- “The indexing strategy was actually decided by another engineer on the team. I mostly handled the cutover.”10:15
- If the cutover had failed mid-migration, what was your rollback plan?10:28
- “We were fairly confident it would work.”10:41
- You said you cut inference cost 60% via quantization. Across how much of the stack?10:50
- “Honestly the quantization only covered our largest model family. I didn't get to the rest before I left.”11:10
Scored only on what was answered. Checked against the transcript.
Built to be forwarded internally without editing.