• Engagement: Contract, paid hourly. No 40-hour minimum — work arrives in sprints, and you take on
what suits you.
• Start: Immediately. The client has sample work that needs turning around this week, with a
substantially larger programme expected to follow.
About the work
Our client builds high-fidelity datasets used to train and evaluate frontier large language models. One of
the things the AI labs want most right now is software engineering tasks for reinforcement learning
environments — specifically, tasks that frontier models cannot solve.
The RL environments are already built. What the client needs is a steady supply of tasks to run inside
them, written by engineers who know what hard, real work actually looks like.
What you’ll be doing
• Writing realistic software engineering tasks: go into a repository, change or fix something, apply a
patch, clean up afterwards. The emphasis is on work an engineer would genuinely be paid to do not competition-style puzzles or textbook exercises.
• Writing the instructions, test cases and expected outputs tightly enough that there is exactly one
correct interpretation.
• Running frontier models against your tasks and evaluating the responses against rigorous functional
and logical standards.
• Where your expertise sits outside Python, helping build out the RL environment for that language.
The bar
A good task defeats a frontier model on the merits: the model understood exactly what was being asked
and still could not do it.
A task that "wins" because the wording was loose does not count. If a model returns something you
weren’t looking for because you didn’t specify properly, that is a fault in the task, not in the model.
Holding that line — hard, but scrupulously unambiguous — is most of the job.
What the client is looking for
• Real engineering depth. The kind that comes from shipping and maintaining production systems.
Years on a CV matter less than the quality of your judgement; some of the client’s strongest
contributors are very young.
• Python, or a very good reason not to. Python is the primary environment. But the client is language
agnostic: if you have spent a career in Java, C++ or COBOL and never picked up Python, that
expertise is genuinely valuable — there is real demand for tasks that move legacy code into modern
languages.
• Intellectual honesty. You say so when you don’t know something, you raise problems while they are
still fixable, and when you say you’re nearly done, you’re nearly done. This matters more than
almost anything else.
• Clear written English. Nearly everything you produce is written specification that someone else has
to be able to read without asking you a question.
Helpful, but not required
• Prior work on AI data platforms — Outlier, Alignerr, Scale, Surge or similar RL and data-annotation
workflows.
• Experience across several codebases, stacks or domains rather than one.
Audio Annotation Trainer - Italian
Careerflow
Freelance Software Developer (Rust) - AI Trainer
Toloka Ai
Analista Senior de Data Analytics - Prevención de Fraude
Lato Jobs
Senior Manager de Riesgos CIB
Bbva
Analista Junior Modelos e Incentivos Banca Empresas
Bbva
Member of Technical Staff - Medical Research (Remote)
Jobs Ai