Retail Specialist | Remote

Retail Subject Matter Expert LLM Training & Evaluation

Shape the reasoning quality of next-generation language models by applying deep retail domain expertise to training data design and structured output evaluation.

$60 $80/hr | 35+ hrs/week | Remote

Key Responsibilities

• Design domain-specific retail tasks spanning merchandising, category management, and operations and produce rigorously reasoned, ground-truth solutions for model training pipelines.
• Evaluate LLM outputs against structured rubrics, delivering precise written assessments of correctness, judgment quality, and reasoning coherence.
• Identify knowledge gaps in model reasoning and guide research and engineering teams toward targeted improvements in retail domain coverage.
• Develop and continuously refine scoring rubrics and evaluation guidelines calibrated to the nuances of real-world retail practice.
• Ensure consistency and accuracy across training datasets by collaborating with peer SMEs during rubric development and cross-review cycles.

Core Requirements

• Extensive hands-on experience in retail with demonstrated progression through roles such as Category Manager, Senior Manager, or Director-level within merchandising, buying/planning, or retail operations at a recognized top-tier retailer.
• Mandatory prior experience evaluating LLM or automation model outputs against structured rubrics or scoring criteria applicants must describe this experience directly in their submission.
• Proven ability to translate practitioner-level retail judgment into written, reproducible evaluation logic suitable for structured training workflows.
• Track record of career advancement within a recognized retail organization (e.g., national or global retailer, major e-commerce platform, or equivalent).
• Exceptional written communication skills with the capacity to articulate nuanced reasoning errors clearly and concisely for technical and non-technical stakeholders.

Additional Strengths

• Background in category analytics, assortment strategy, or demand planning adds meaningful depth to task and rubric design.
• Familiarity with annotation platforms, data quality workflows, or structured evaluation frameworks is a strong differentiator.
• Experience operating within extended workforce, contract, or embedded-team environments signals readiness for this engagement model.

Application Process

• Apply through the Easy Apply button to submit your application.
• Applications will be carefully reviewed in line with the role requirements.
• Eligible applicants will be contacted via email with instructions for the next stage.
• Follow the instructions in the email to complete the remaining application requirements.