Remote Data Labeling Specialist (Vancouver)

Remote Data Labeling Specialist (Vancouver) — Full-Time

Rex.zone is hiring a Remote Data Labeling Specialist to deliver consistent, policy-aligned annotations used to train and evaluate AI/ML models. You will work across NLP labeling, RLHF preference ranking, prompt evaluation, computer vision annotation, and content safety labeling to improve training data quality and downstream model performance.

Key Responsibilities

• Apply annotation guidelines accurately across NLP and computer vision tasks
• Perform RLHF preference ranking and rubric-based evaluations to align model behavior with human preferences
• Execute prompt evaluation and response quality scoring for LLM evaluation workflows
• Conduct QA evaluation, sampling audits, guideline compliance checks, and error categorization to reduce label noise
• Label content safety categories (e.g., harassment, self-harm, hate) using policy taxonomies
• Collaborate with project leads to resolve edge cases, refine rubrics, and document decisions
• Track productivity and accuracy metrics to support model performance improvement
  
  

Workstreams You May Support

• NLP data labeling: named entity recognition (NER), intent classification, sentiment/topic labeling, summarization evaluation
• LLM training pipelines: instruction-following evaluation, pairwise preference ranking, rubric grading
• Computer vision annotation: bounding boxes, polygons, keypoints, segmentation, OCR validation
• Content safety labeling: policy-based moderation labels and severity scoring
• Multimodal tasks: text-image relevance, caption validation, VQA-style evaluation
  
  

Required Qualifications

• Experience applying structured guidelines to data annotation and/or QA evaluation work
• Strong written communication for consistent labeling rationale and disagreement resolution
• Comfort working with ambiguity, edge cases, and evolving rubrics
• Familiarity with NLP, RLHF, or LLM evaluation concepts (training data quality, preference ranking, prompt evaluation)
• Ability to maintain accuracy at production scale and meet full-time delivery expectations
  
  

Preferred Qualifications

• Experience with annotation platforms and audit workflows
• Exposure to computer vision annotation tools (boxes, polygons, segmentation)
• Experience with content safety labeling or policy taxonomy interpretation
• Experience documenting guidelines, building QA checklists, or running inter-annotator agreement checks
  
  

Remote Work Notes

This is a Remote role. You can be based in Vancouver while working with distributed teams. Projects may require overlap with US business hours, and task availability can vary by domain.

Pay: $30–$50 per hour (base salary).