Remote Data Annotation Jobs in Lisbon

About The Role

Remote data annotation work supporting AI/ML training pipelines via Rex.zone, including LLM instruction tuning, RLHF preference ranking, prompt evaluation, and QA evaluation. This role is recruited with Lisbon intent and delivered fully remotely through online tools and distributed calibration workflows.

What You Will Do

• Label and validate training data across NLP and computer vision tasks (text, images, structured items)
• Evaluate LLM outputs using rubric-based scoring, prompt evaluation, and response grading
• Perform pairwise preference ranking to generate RLHF signals
• Run QA evaluation checks, sampling audits, and disagreement analysis to reduce label noise
• Document edge cases and provide rationale notes to improve guidelines and consistency
• Maintain annotation guidelines compliance and contribute to training data quality improvements
  
  

Tools, Data Types, and Methods

• Web-based annotation platforms and structured task queues
• Datasets such as chat transcripts, prompts, policy documents, product text, and images
• Calibration against gold tasks, inter-annotator agreement checks, and error taxonomy tagging
  
  

Required Qualifications

• Mid-to-senior experience in data annotation, data labeling, or AI/ML evaluation
• Strong written communication for QA feedback and edge-case rationale
• Familiarity with LLM evaluation, prompt evaluation, and/or RLHF-style ranking
• Quality-first mindset in remote, metric-driven production environments
  
  

Work Setup

Remote full-time role with asynchronous execution and scheduled calibration sessions across time zones.

Compensation

Hourly base pay range: $30–$50 per hour (USD), depending on project needs and calibration performance.