Senior Data Annotator Jobs in London

Senior Data Annotator (Remote, Full-time)

Join Rex.zone to produce high-quality training data for AI/ML systems. You will lead annotation workstreams for LLM training pipelines, generating gold-standard labels, RLHF preference data, and QA evaluation outputs to improve downstream model performance across NLP, computer vision, and content safety workflows.

What You Will Do

• Execute and review data annotation tasks across text, image, and multimodal datasets
• Create and refine labeling taxonomies, edge-case definitions, and decision rubrics
• Perform RLHF preference ranking, pairwise comparisons, and rubric-based scoring
• Run QA evaluation, error analysis, and inter-annotator agreement checks
• Complete prompt evaluation for instruction-following, helpfulness, and harmlessness
• Label and validate named entity recognition datasets and entity linking rules
• Contribute to content safety labeling (toxicity, self-harm, hate, policy compliance)
• Maintain annotation guidelines compliance documentation and change logs
• Collaborate with engineering and data operations to improve tooling and throughput
  
  

Requirements

• Mid-senior experience in data annotation, data labeling, or AI/ML evaluation
• Experience with RLHF workflows and rubric-based evaluation
• Strong writing and reasoning skills for LLM evaluation and prompt evaluation
• Comfort with QA evaluation methods (sampling, audits, consistency checks)
• Familiarity with structured schemas such as named entity recognition
• Ability to work independently in a remote environment with high attention to detail
  
  

Success Metrics

• Improved training data quality (audit pass rates, lower defect rates)
• High agreement and stable calibration across annotators/reviewers
• Clear documentation of edge cases, rubric updates, and QA outcomes
  
  

Pay

$30–$50 per hour (base pay; rate depends on skills and experience).