Member of Technical Staff: Python, TypeScript & LLM Agent Evals | San Francisco On-site | Up to $400K + Equity

"Good morning! And in case I don't see ya..." Truman Burbank, The Truman Show

Every AI agent thinks it lives in a perfect little world. Clean prompts, polite users, happy paths only. Sunny Seahaven, every day.

Your job is to sail out to the horizon and find where the sky is painted on.

THE OPPORTUNITY

A well-funded, seed-stage San Francisco startup is building the trust layer for AI agents, making sure they're safe and reliable before they're handed real responsibility in high-stakes industries. It's backed by top-tier AI investors, and the team is small, senior and stacked with people who have built companies before.

These agents are already in production at some of the most widely used AI products on the planet. Somebody has to find out how they break before the real world does. That's you.

This is a senior individual contributor seat with a rare mix: hardcore engineering, frontier AI research turned into product, and real contact with the customer teams building the agents everyone uses.

WHAT YOU'LL ACTUALLY DO

• Own red-team engagements end to end on major customers' agents: scoping, recon, attack execution, remediation and sign-off
• Sharpen the core engine that powers every evaluation the company runs
• Turn bleeding-edge adversarial research into repeatable product
• Be the palate for eval quality. If a test is weak, you send it back to the kitchen
• Automate your own workflow with agents, because doing it twice by hand is a crime
• Work out how to test the next wave, like computer-use agents, before anyone has a playbook for them

THE TECH YOU'LL LIVE IN

Python is the engine room. Evals, attack pipelines and grading logic all run here, so you'll write production-grade Python fluently. That means async code that fires thousands of conversations at an agent in parallel, clean typed data models, solid tests, and the judgment to know when a script needs to grow up into a system.

TypeScript is where the work meets people. Customers need to see results, findings and reports, and you'll ship that user-facing side yourself. If you're strong in React and Node, typed end to end and built with the same care as the backend, you'll feel at home. This one is non-negotiable: they want engineers who already write both languages well, not people planning to learn one.

LLMs and agents are your raw material. You should be comfortable with the major model APIs, tool and function calling, structured outputs, retrieval, multi-turn agent loops, and protocols like MCP that connect agents to the outside world. Hands-on time with agent frameworks and orchestration is a big plus.

Evals are the craft. Designing test suites that actually catch failures. Using LLM-as-judge grading without fooling yourself. Building golden datasets, generating synthetic adversarial scenarios at scale, and proving whether a fix really worked or just moved the problem somewhere else.

Adversarial AI is the fun bit. Prompt injection, jailbreaks, tool misuse, data leakage, harmful outputs, and agents going off-script under multi-turn pressure. If you've read the latest red-teaming papers and itched to try them on a real production system, this is the job.

Integration is the glue. Every customer's agent is different, so you'll work across SDKs, API endpoints, webhooks and the occasional ugly legacy system, often getting a new integration running in days. Browser automation and computer-use tooling will matter more and more as agents start clicking around the web on their own.

MUST HAVE

• Real production experience shipping user-facing features, not just backend plumbing
• Strong, hands-on Python and TypeScript. Both, not "happy to pick it up"
• 5+ years as a software engineer, owning high-stakes technical work from start to finish
• A career arc that keeps climbing in title and scope
• A CS degree or equivalent technical background
• Agentic AI tools baked into how you work every day
• Clear communication with engineers and non-technical stakeholders alike
• On-site in San Francisco, 5 days a week

BONUS POINTS

• Founder, founding engineer or early startup hire (extra credit if it was VC-backed)
• Customer-facing experience
• Hands-on with agentic frameworks and evals
• A visible spike: a side project people talk about, open source, writing, a standout company, or a bold career move that shows real conviction

PROBABLY NOT FOR YOU IF

• Your whole career has been at big tech with no early-stage time
• You're an engineering manager who's drifted away from writing code
• You're after a management or director title. This is a senior IC seat, and that's the point

WHY THIS ONE IS WORTH YOUR TIME

• You'll work on one of the biggest open problems in AI: knowing whether an agent can be trusted before it does something that matters
• Your work lands in production with customers whose agents are used by millions
• Small team, huge surface area, and a direct line to the founders
• The category is brand new, so the playbook you write could become the industry standard

THE PACKAGE

• Up to $400K base + meaningful equity
• On-site in SF with a small, sharp, very senior team
• Visa transfers welcome (OPT, H-1B transfers)
• A quick, focused process: a short vibe check, a coding round, then an onsite covering systems design, algorithms and a deep-dive on AI evals

HOW TO APPLY

Hit apply with your resume and answer the screening questions. Big Wave Digital is recruiting this role on behalf of our client, and every application is read by a real person.

"We accept the reality of the world with which we're presented." Christof, The Truman Show

The agents accept it. You won't.