Research Scientist

About the Role

Join a research and engineering team at an early-stage AI company focused on model evaluation and benchmarking. You will help develop practical methods for testing language models and other AI systems, shaping how their capabilities are assessed before deployment.

What You'll Do

• Evaluate newly released AI models across varied use cases and assess their capabilities.

• Develop benchmarks, including building datasets, coordinating labeling work, and contributing to research publications.

• Advance automated evaluation methods for generated text to improve scalability and accuracy.

• Work with engineering teams to implement and scale evaluation methods in production systems.

• Collaborate with AI labs and enterprise customers to understand evaluation needs and turn them into research initiatives.

What We're Looking For

• AI/ML research experience, including experience with benchmarking, evaluation methods, or language models. Candidates with 0 to 3 years of applied AI experience may be considered.

• Strong Python skills and experience writing clean, maintainable code, ideally with Django, Flask, or another Python HTTP framework.

• Hands-on experience with PyTorch or TensorFlow, and familiarity with LLM infrastructure, language modeling, and diffusion models.

• Experience developing benchmarks or evaluation methods; NLP research and relevant publications are valuable.

• A degree in Computer Science, Machine Learning, or a related field is preferred at the master's or doctoral level. Candidates with a bachelor's degree and 1 to 3 years of relevant experience may also be considered.

• Strong collaboration and communication skills, including experience with Git workflows, code review, and incorporating feedback.

Compensation & Benefits

Base salary is $150,000 to $200,000 annually. Visa sponsorship is available.

Location

On-site in San Francisco, California, United States.