AI Engineer

About Us

Solumn AI helps frontier AI labs understand and improve how their models behave in challenging, real-world environments. We build the evaluation environments, research infrastructure and methods that test AI systems against frontier labs’ safety and alignment standards, and we turn that research into evaluations and training data that measurably improve model behaviour.

About the job

We’re looking for AI Engineers who combine solid engineering with research judgement. You’ll investigate how frontier models behave, form hypotheses, design experiments, and build the environments, benchmarks and datasets that produce reliable evidence. This goes beyond implementing existing benchmarks: you’ll help decide what to measure, how to test it rigorously, and how to turn open-ended questions into reproducible systems.

How you will contribute

• Research and experimental design: turn safety questions into hypotheses, controlled experiments, baselines and clear measurement criteria
• Evaluation environments: build realistic agentic and tool-use environments, execution harnesses, trajectory collection and scalable experiment workflows
• Benchmarks and evaluation quality: create benchmark datasets, robust graders and verifiers, and QA that catches shortcuts and misleading scores
• Model behaviour analysis: investigate safety-relevant behaviours and failure modes, and tell genuine model limitations apart from environment, rubric or verifier defects
• Engineering standards: improve reproducibility, test coverage, documentation and internal tooling across research projects

Who you are

• You write strong Python and have built and debugged APIs, pipelines, tools or other complex systems
• You understand LLMs, transformers, inference, tool use and agentic architectures; post-training or RL experience is a plus
• You can read technical work critically, design experiments and interpret results; a publication record is not required
• You care about evaluation validity, confounders, reproducibility and verifier robustness
• You’re comfortable with ambiguity, learn fast, work independently, and are genuinely interested in AI safety and alignment

Bonus if you’ve worked on

• LLM evaluations, benchmark development or model behaviour analysis
• Agentic, multi-agent or tool-use systems
• AI safety, red teaming, adversarial testing or security research
• ML research engineering, experiment infrastructure or automated grading
• Dataset development, synthetic data or post-training workflows
• Full-time · 6-day work week · 100% Remote