Senior Data Annotation Specialist

Senior Data Annotation Specialist (Remote, Full-Time)

Rex.zone is hiring senior data annotation specialists to lead complex labeling and evaluation workflows that produce reliable training data for AI/ML systems. You will contribute to large language model evaluation, RLHF preference ranking, rubric-based QA evaluation, and prompt evaluation to improve downstream model performance.

What You Will Do

• Execute senior-level data labeling and review for text, image, and multi-modal datasets
• Perform RLHF ranking, preference labeling, and rationale writing aligned to evaluation rubrics
• Run QA evaluation cycles: spot checks, double-pass review, adjudication, and error taxonomy
• Perform prompt evaluation for helpfulness, correctness, safety, and instruction-following
• Maintain annotation guidelines compliance; propose clarifications and decision rules
• Track training data quality metrics (accuracy, agreement, coverage) and drive improvements
• Support dataset governance: versioning notes, issue logs, and escalation of ambiguous cases
• Collaborate with cross-functional partners supporting LLM training pipelines (ops, QA, engineering)
  
  

Required Qualifications

• Professional experience in data annotation, data labeling, or QA evaluation for AI/ML
• Demonstrated capability with rubric-based judgment tasks and edge-case reasoning
• Familiarity with large language model evaluation and RLHF-style workflows
• Strong written communication for rationales, guideline updates, and audit notes
• High attention to detail and consistent application of policy and annotation guidelines
• Comfort working with structured tooling, queues, and quality sampling processes
  
  

Domains You May Support

• NLP tasks (named entity recognition, classification, summarization)
• Computer vision annotation (bounding boxes, polygons, attributes)
• Content safety labeling aligned to policy and safety rubrics
  
  

Compensation

Base pay range: $30–$50 per hour (HOURLY).

Benefits may include health coverage options, paid time off, and remote-work support depending on engagement terms and location.

How To Apply

Apply via Rex.zone with a resume highlighting senior data annotation, RLHF, QA evaluation, and guideline compliance experience, plus examples of training data quality improvements.