Software Engineer, RL Environments

About the Role

Design datasets, evaluation rubrics, and reward signals that help improve frontier AI models. You will work with research teams to turn training goals into practical experiments and data specifications, with results feeding into model training runs.

What You'll Do

• Design data slices across finance, software, and enterprise workflows to reveal meaningful model failure modes.

• Build and refine evaluation rubrics and reward signals for RLHF and RLVR pipelines.

• Model annotator behavior and run experiments to improve model capabilities.

• Develop quantitative approaches to measure dataset quality, diversity, and downstream impact.

• Create and manage real-world and synthetic data pipelines.

• Partner with research teams to translate training objectives into data and evaluation specifications.

What We're Looking For

• One to four years of hands-on experience designing machine learning data collection strategies and evaluation rubrics.

• Strong familiarity with LLM training and evaluation methods, including experience creating data slices to surface model failure modes.

• Ability to run lightweight experiments, work through ambiguous results, and extract actionable insights.

• Comfort working across domains such as finance, software engineering, and policy.

• A practical, builder-oriented approach and a strong interest in how data structure and quality shape model behavior.

Compensation & Benefits

Base salary: $260,000 to $290,000 annually. Visa sponsorship is available.

Location

On-site in San Francisco, California.