Senior Data Labeling Jobs in Warsaw (Remote)

Overview

Senior Data Labeling jobs in Warsaw (Remote) at Rex.zone focus on building high-quality training datasets for modern AI systems. In this Mid-Senior, FULL_TIME remote role, you will lead labeling workflows across LLM training pipelines, RLHF evaluation, prompt evaluation, content safety labeling, and QA evaluation to drive measurable model performance improvement.

What You Will Do

• Lead end-to-end data labeling operations for NLP and computer vision programs, ensuring training data quality and consistency
• Execute and review RLHF tasks (preference ranking) and rubric-based evaluations for LLM outputs
• Run QA evaluation methods including audit sampling, gold set management, and inter-annotator agreement analysis
• Enforce annotation guidelines compliance, calibrate raters, resolve edge cases, and maintain decision logs
• Track error taxonomies and close feedback loops with engineering to improve datasets and evaluation outcomes
• Support content safety labeling workflows using policy-driven severity ratings
  
  

Required Qualifications

• Mid-Senior experience in data labeling / annotation / AI evaluation
• Strong ability to apply and improve annotation guidelines, rubrics, and taxonomies
• Hands-on experience with QA evaluation (audits, gold data, inter-annotator agreement)
• Familiarity with RLHF workflows, preference ranking, prompt evaluation, and LLM evaluation concepts
• Excellent written documentation for edge cases and policy decisions
  
  

Compensation

Hourly base pay range: $30–$50 (USD, HOURLY). Final compensation depends on scope and demonstrated senior-level QA and evaluation capability.

How To Apply

Apply via Rex.zone and highlight RLHF evaluation experience, QA methods, annotation guideline compliance, and examples of improving training data quality.

Remote role aligned with Warsaw search intent.