Machine Learning Engineer - LLM Evaluator - Hybrid Opportunity
- Salary
- $55–$65/hr
- Hiring from
- United States
- Work type
- Hybrid
- Posted
625,482 remote jobs, straight from company career pages
100% free · New jobs every minute
Show job descriptionHide job description
Description
* Hybrid role requiring onsite attendance in Cupertino, CA three (3) days per week.
* Contract opportunity.
* This role focuses on AI/LLM quality evaluation and measurement, not traditional model development.
* Candidates should have experience owning or leading evaluation strategies for production AI/LLM systems.
We are seeking an experienced Machine Learning Engineer to help define, measure, and improve the quality of AI systems powered by Large Language Models (LLMs). In this role, you will design and scale evaluation frameworks that ensure AI-generated outputs are accurate, reliable, and consistent across a variety of use cases.
You will partner closely with Product, Engineering, Research, and Data teams to establish evaluation methodologies, automate quality assessment workflows, and develop the metrics that drive model improvements. This is an ideal opportunity for someone who enjoys solving complex problems at the intersection of AI quality, evaluation strategy, automation, and data-driven decision making.
Top Skills
- AI/LLM Evaluation & Quality Measurement
- Problem Solving & System Design
- Communication & Ownership
- Python, SQL & Automation
Top Responsibilities
- Design and build LLM evaluation frameworks, including automated scoring systems, rubric-based grading, and LLM-as-a-Judge solutions.
- Define and track evaluation metrics, KPIs, and dashboards that measure model quality and performance at scale.
- Perform error analysis, root-cause investigations, and quality deep dives to identify opportunities for model improvement.
- Partner with cross-functional teams to integrate evaluation standards, workflows, and quality requirements into production environments.
What We're Looking For
- Experience designing and maintaining LLM evaluation frameworks and measurement methodologies.
- Hands-on experience with autograders, LLM-as-a-Judge systems, and automated quality validation.
- Strong prompt engineering skills, including the development of evaluation prompts and grading rubrics.
- Proficiency with Python, SQL, and large-scale data analysis.
- Experience creating KPIs, dashboards, and executive-ready reporting on AI quality.
- Ability to evaluate model performance through error analysis, human-vs-model calibration, and quality benchmarking.
- Experience working with production AI/LLM systems and large evaluation datasets.
What Will Make You Stand Out
- You've personally owned or led an AI/LLM evaluation strategy.
- You've built scalable systems to measure AI quality automatically.
- You can clearly explain how AI quality is defined, measured, and improved.
- You have experience influencing product and engineering decisions through evaluation insights and data.
Experience Level: Expert
Job Type & LocationThis is a Contract position based out of Cupertino, CA.
Pay and BenefitsThe pay range for this position is $55.00 - $65.00/hr.
Individual compensation offered for this position within this range will depend on many factors, including qualifications, skills, relevant experience, job knowledge, geographic location, internal equity, and other pertinent job-related factors.
Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits available for this temporary role may include the following: Medical, dental & vision Critical Illness, Accident, and Hospital 401(k) Retirement Plan Pre-tax and Roth post-tax contributions available Life Insurance (Voluntary Life & AD&D for the employee and dependents) Short and long-term disability Health Spending Account (HSA) Transportation benefits Employee Assistance Program Time Off/Leave (PTO, Vacation or Sick Leave)
Job Type & LocationThis is a Contract position based out of Cupertino, CA.
This position is anticipated to close on Nov 20, 2026.
Workplace TypeThis is a fully remote position.
About TEKsystems
We're partners in transformation. We help clients activate ideas and solutions to take advantage of a new world of opportunity. We are a team of 80,000 strong, working with over 6,000 clients, including 80% of the Fortune 500, across North America, Europe and Asia. As an industry leader in Full-Stack Technology Services, Talent Services, and real-world application, we work with progressive leaders to drive change. That's the power of true partnership. TEKsystems is an Allegis Group company.
The company is an equal opportunity employer and will consider all applications without regards to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.
About TEKsystems and TEKsystems Global Services
Were a leading provider of business and technology services. We accelerate business transformation for our customers. Our expertise in strategy, design, execution and operations unlocks business value through a range of solutions. Were a team of 80,000 strong, working with over 6,000 customers, including 80% of the Fortune 500 across North America, Europe and Asia, who partner with us for our scale, full-stack capabilities and speed. Were strategic thinkers, hands-on collaborators, helping customers capitalize on change and master the momentum of technology. Were building tomorrow by delivering business outcomes and making positive impacts in our global communities. TEKsystems and TEKsystems Global Services are Allegis Group companies. Learn more at TEKsystems.com.
The company is an equal opportunity employer and will consider all applications without regard to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.
San Francisco Fair Chance Ordinance: Pursuant to the San Francisco Fair Chance Ordinance, for all positions located in the city and county of San Francisco, we will consider for employment qualified applicants with arrest and conviction records.
Massachusetts Lie Detector: It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.
Use of Artificial Intelligence (AI): We may use Artificial Intelligence (AI) to support parts of our hiring process, including sourcing, screening, and evaluating candidates. AI helps assess applications and qualifications, but final decisions are made by our hiring team. By applying, you acknowledge and agree that your application may be reviewed using AI tools.
