Senior Data Annotator Jobs in Dublin

Senior Data Annotator (Remote, Full-Time)

Join Rex.zone to support AI/ML training workflows by producing and reviewing high-quality labeled data used for model training and evaluation. You will execute guideline-driven annotation, QA evaluation, RLHF preference judgments, and prompt evaluation to strengthen LLM training pipelines across NLP, computer vision, and content safety.

Key Responsibilities

• Execute senior-level data labeling and review for text, image, and multimodal datasets
• Perform RLHF preference ranking and large language model evaluation using defined rubrics
• Run QA evaluation, audits, and calibration rounds to ensure training data quality
• Identify guideline gaps, document edge cases, and propose rubric clarifications
• Monitor inter-annotator agreement and resolve disagreements through calibration
• Support prompt evaluation for helpfulness, harmlessness, and policy compliance
• Contribute to content safety labeling and sensitive-topic handling processes
• Partner with engineering and data ops to improve tooling, throughput, and error detection
  
  

Required Qualifications

• Experience in data annotation, data labeling, or dataset QA for ML systems
• Comfort applying detailed rubrics and maintaining annotation guidelines compliance
• Familiarity with RLHF concepts, preference data, or LLM evaluation workflows
• Strong written reasoning for judgment-based tasks and edge-case documentation
• Experience with NLP (e.g., named entity recognition) and/or CV annotation (e.g., bounding boxes, segmentation)
  
  

How Quality Is Measured

Quality is assessed via audits, inter-annotator agreement checks, rubric adherence, calibration rounds, and error-rate monitoring to protect downstream model performance.

Pay

$30–$50 per hour (USD)