Full-Stack Engineer, Agent Evaluation & Reliability

About the Role

Join an AI/ML engineering team building infrastructure for agent evaluation and continuous improvement. This backend-leaning full-stack role combines foundational systems work with end-to-end product delivery, helping make agent workflows more reliable for enterprise customers.

What You'll Do

• Build backend systems that power agent evaluation and continuous-improvement workflows.

• Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings, and customer-defined quality criteria.

• Build and maintain enterprise integrations, including authentication, OAuth, webhooks, incremental syncs, backfills, rate-limit handling, retries, and monitoring.

• Own features from technical design and implementation through deployment, monitoring, customer feedback, and iteration.

• Work with customers to understand their systems, diagnose failures, and turn specific needs into reusable product capabilities.

• Take features through the full stack while contributing to architecture and product decisions.

What We're Looking For

• Two to four years of experience in a full-stack or backend engineering role, building and shipping production systems.

• Production experience with TypeScript and Node.js or Bun.

• Hands-on machine learning model development experience, in addition to software engineering experience.

• Experience at a fast-paced startup, from early stage through Series D.

• Production experience designing PostgreSQL schemas and working with queues or worker systems.

• A track record of owning projects end to end and building backend systems involving data modeling, integrations, observability, or distributed workflow debugging.

• Understanding of how evaluating nondeterministic agent systems differs from conventional unit testing.

• Experience with agent infrastructure, LLM evaluation, tracing, or agent memory systems in production or through substantial side projects is useful.

Compensation & Benefits

Base salary is $120,000 to $180,000 annually. Visa sponsorship is available, including H-1B transfers, OPT, and O-1.

Location

On-site in San Francisco, California, United States.