Senior Data Annotator Jobs in Warsaw

Senior Data Annotator Jobs in Warsaw (Remote)

Rex.zone is hiring senior data annotators to create and validate high-quality training data for modern AI/ML systems. You will support LLM training pipelines through data labeling, QA evaluation, prompt evaluation, and RLHF-style preference ranking to improve model behavior and evaluation quality.

What You Will Do

• Annotate and review datasets across NLP, computer vision, and content safety domains
• Perform LLM evaluation tasks such as pairwise ranking, multi-turn conversation grading, and preference data creation (RLHF)
• Execute QA evaluation via audits, defect pattern reporting, and inter-annotator agreement checks
• Conduct named entity recognition (NER), taxonomy tagging, and consistent label application
• Evaluate prompts and model outputs for helpfulness, harmlessness, factuality, and policy compliance
• Collaborate with leads to refine guidelines, document edge cases, and improve calibration
  
  

Requirements

• Experience in data annotation/data labeling with strong quality outcomes
• Excellent annotation guidelines compliance and consistency on edge cases
• Experience with LLM evaluation, prompt evaluation, or RLHF-style ranking
• Strong written documentation and feedback skills
• Comfort working remotely with throughput, accuracy, and audit expectations
  
  

How Quality Is Measured

• Audit scores and guideline adherence
• Inter-annotator agreement and reviewer calibration
• Consistency and clarity in edge-case documentation
  
  

How To Apply

Apply via Rex.zone and include your annotation domains (NLP, computer vision annotation, content safety labeling) and examples of QA evaluation, guideline improvements, or RLHF/LLM evaluation work.