Senior Data Annotation Specialist (Remote, Full-Time)
Rex.zone is hiring senior data annotation specialists to lead complex labeling and evaluation workflows that produce reliable training data for AI/ML systems. You will contribute to large language model evaluation, RLHF preference ranking, rubric-based QA evaluation, and prompt evaluation to improve downstream model performance.
What You Will Do
Execute senior-level data labeling and review for text, image, and multi-modal datasets
Perform RLHF ranking, preference labeling, and rationale writing aligned to evaluation rubrics
Run QA evaluation cycles: spot checks, double-pass review, adjudication, and error taxonomy
Perform prompt evaluation for helpfulness, correctness, safety, and instruction-following
Maintain annotation guidelines compliance; propose clarifications and decision rules
Track training data quality metrics (accuracy, agreement, coverage) and drive improvements
Support dataset governance: versioning notes, issue logs, and escalation of ambiguous cases
Collaborate with cross-functional partners supporting LLM training pipelines (ops, QA, engineering)
Required Qualifications
Professional experience in data annotation, data labeling, or QA evaluation for AI/ML
Demonstrated capability with rubric-based judgment tasks and edge-case reasoning
Familiarity with large language model evaluation and RLHF-style workflows
Strong written communication for rationales, guideline updates, and audit notes
High attention to detail and consistent application of policy and annotation guidelines
Comfort working with structured tooling, queues, and quality sampling processes
Content safety labeling aligned to policy and safety rubrics
Compensation
Base pay range: $30–$50 per hour (HOURLY).
Benefits may include health coverage options, paid time off, and remote-work support depending on engagement terms and location.
How To Apply
Apply via Rex.zone with a resume highlighting senior data annotation, RLHF, QA evaluation, and guideline compliance experience, plus examples of training data quality improvements.