Senior Data Labeling jobs in Warsaw (Remote) at Rex.zone focus on building high-quality training datasets for modern AI systems. In this Mid-Senior, FULL_TIME remote role, you will lead labeling workflows across LLM training pipelines, RLHF evaluation, prompt evaluation, content safety labeling, and QA evaluation to drive measurable model performance improvement.
What You Will Do
Lead end-to-end data labeling operations for NLP and computer vision programs, ensuring training data quality and consistency
Execute and review RLHF tasks (preference ranking) and rubric-based evaluations for LLM outputs
Run QA evaluation methods including audit sampling, gold set management, and inter-annotator agreement analysis
Track error taxonomies and close feedback loops with engineering to improve datasets and evaluation outcomes
Support content safety labeling workflows using policy-driven severity ratings
Required Qualifications
Mid-Senior experience in data labeling / annotation / AI evaluation
Strong ability to apply and improve annotation guidelines, rubrics, and taxonomies
Hands-on experience with QA evaluation (audits, gold data, inter-annotator agreement)
Familiarity with RLHF workflows, preference ranking, prompt evaluation, and LLM evaluation concepts
Excellent written documentation for edge cases and policy decisions
Compensation
Hourly base pay range: $30–$50 (USD, HOURLY). Final compensation depends on scope and demonstrated senior-level QA and evaluation capability.
How To Apply
Apply via Rex.zone and highlight RLHF evaluation experience, QA methods, annotation guideline compliance, and examples of improving training data quality.