We are helping our client find exceptional
Software Engineer - Technical Advisors
who want to spend a focused period doing something most engineers never get to do: find out exactly where frontier models break on real engineering work.
You will take on hard problems across production-grade codebases, then work out precisely where and why model-generated solutions fall short. You will also build the hard problems those models are tested on. Your judgment about what separates good engineering from plausible engineering is the entire point of the role.
What you'll do
Evaluate Agent Sessions: Analyze how AI coding agents behave over entire coding sessions—tracking what the agent investigates, verifies, assumes, and leaves undone.
Review Model-Written Pull Requests: Audit model-generated pull requests against real production repositories, documenting every identified issue alongside its severity and detailed technical rationale.
Build Evaluation Benchmarks: Design and construct hard, container-based problems that serve as rigorous test benchmarks for frontier models.
Collaborate with AI Researchers: Work directly alongside AI researchers on frontier problems, producing clear written analyses that explain root-cause failures and boundary-condition gaps.
Maintain High Written Rationale Standards: Author original, clear technical rationale for all evaluations (all written deliverables must be independently authored without AI text generation, though AI tools are welcomed for codebase exploration and running test suites).
Requirements:
8+ years of production engineering experience
preferred (exceptions only for clearly exceptional profiles).
Backgrounds including
Senior, Staff, or Principal Software Engineers, Tech Leads, or Open-Source Maintainers
.
Experience working in production codebases with a strict
code review culture
(startups, big tech, or open source are all acceptable).
Polyglot adaptability:
Heavy experience with
Python and TypeScript
is common, but must be comfortable dropping into unfamiliar languages weekly.
Core Tooling:
Demonstrated comfort using Docker, git, and the Command Line Interface (CLI) to reproduce, isolate, and debug results locally.
Cross-Layer Fluency:
Comfort working across backend, frontend, APIs, data, testing, or developer tooling.
Strong written communication skills—able to articulate
why
code fails, not just how to fix it quickly.