Transluce logo

Research Scientist - Evaluations

Salary
$310K–$500K
Moves you to
United States
Support
Visa sponsorship
Posted

75,578 relocation jobs, straight from company career pages

100% free · New jobs every minute

Show job description
Salary range: $310,000 - $500,000/year + benefits

Description: Transluce is a fast-moving nonprofit research lab building the public tech stack for AI evaluation and oversight. We partner with frontier AI developers, governments, and civil society organizations to evaluate how AI systems behave, from misaligned agent swarms to child safety and mental health. Our research has uncovered surprising behaviors in frontier models, such as pre-release models fabricating actions they never took. Our evaluations directly underpin policy, including through collaborations with the EU AI Office and Common Sense Media.

About the role: We’re looking for research scientists to work on measuring AI model behaviors and developing new methodologies for evaluating AI systems. You will build independent evaluations of AI models and do research into key questions that guide that work: how can we discover problems with AI models before they cause negative impacts in the world? How can we leverage AI assistance to automate parts of the evaluation process?

You’ll help to set standards for our evaluation work across the board and work with engineers to translate research insights into reusable tools. For example, you might develop a new methodology for discovering interesting LLM behaviors from in-the-wild transcripts; collaborate with other teams to build a scalable pipeline for behavior discovery; and, publish your findings and tools to advance evaluation work here and in other organizations.

This is a high-autonomy role with a clear line of sight from your work to real-world impact. As part of a mission-focused non-profit, you’ll contribute to strengthening independent oversight of AI and advancing the science of AI evaluation.

Core responsibilities: Develop evaluation methods for measuring evolving AI model behaviors and put those methods into practice by building public evaluations. This includes:
  • Researching techniques for building realistic simulations or reproductions of AI deployment environments.
  • Developing high quality public evaluations to raise the bar for evaluation work across the independent oversight ecosystem.
  • Informing and contributing to the development of tools for automating parts of the evaluation process.
  • Shaping Transluce’s approach to evaluations and investigating fundamental questions in evaluation science.

Qualities of a strong candidate:
  • Hands-on experience designing and running AI evaluations, particularly behavioral or interactive evaluations (multi-turn, agentic, or red-teaming contexts).
  • Research track record that includes careful empirical or methods work, ideally in evaluation science.
  • Experience working in cross functional teams of researchers, engineers, and domain experts.
  • Strong communication skills, low ego, openness to giving and receiving feedback.

Preferred qualifications (nice to have):
  • Engineering skills: implementing well-designed software, user testing, and iterating on feedback.
  • Past work on building agent harnesses or LLM-powered automated research systems.

We are located in San Francisco and enthusiastic to work together in-person. We are open to sponsoring international visas.

Similar jobs

Apply for this job100% free