AT
AI Engineer — Retrieval & Evaluation
- Hiring from
- India
- Work type
- Remote
- Posted
Is this job info correct?
604,628 remote jobs, straight from company career pages
100% free · New jobs every minute
Show job descriptionHide job description
athenaprep ResourcesOur StoryGuideBlogProfilePricing
HelpdeskGo to Dashboard
ResourcesOur StoryGuideBlogProfilePricingHelpdeskGo to Dashboard
All open roles
AIActively hiringPosted 3 months ago
AI Engineer — Retrieval & Evaluation
Make Ask Athena genuinely trustworthy. This is retrieval over our own question bank and a student's own notes — where a confidently wrong answer costs someone marks.
Apply for this role
Experience 2–5 yrs
Location Remote (India)
Arrangement Remote
Employment Full-time
Department AI
Responsibilities
Nice to have, not required — apply even if none of these are true of you.
How we'll judge this role together — shared with you on day one, not at review time.
Stand up the evaluation harness and establish the quality baseline.
Own the full AI surface — retrieval, grounding and cost.
Define AI strategy as the product's intelligence layer deepens.
Skills you'll use
Hiring timeline
We reply within 3 days
A short note on why this problem interests you beats a long CV. Links to things you've made help most.
30 minutes
A relaxed call about your work, what you want next, and what we're actually building. Questions go both ways.
Take-home or live
A small, time-boxed piece of work close to what the role actually involves. No trick puzzles, no unpaid project work.
45 minutes
Values, ownership, and how you'd want the first ninety days to go.
Within a week
Clear terms, clear scope, and a start date that works for you.
Questions
Yes. We care much more about evidence of craft than years of experience — a project you built and can explain in depth counts for more than a familiar company name.
Yes, remote-first across India, and async by default. We write decisions down so nobody needs to be online at a particular hour to stay in the loop.
Usually under three weeks from application to offer. We reply to every application within three days, even when the answer is no.
Yes, and they are real work with real ownership, not shadowing. Strong interns are regularly offered full-time roles.
Join the talent community below. We open roles as we grow, and the people already in the pool are the first ones we contact.
Think this is you?
A short note on why this problem interests you beats a long CV. If you're close but not an exact match, apply anyway — we'd rather read your note than miss you.
Apply for AI Engineer — Retrieval & Evaluation Join talent community
Apply now
HelpdeskGo to Dashboard
ResourcesOur StoryGuideBlogProfilePricingHelpdeskGo to Dashboard
All open roles
AIActively hiringPosted 3 months ago
AI Engineer — Retrieval & Evaluation
Make Ask Athena genuinely trustworthy. This is retrieval over our own question bank and a student's own notes — where a confidently wrong answer costs someone marks.
Apply for this role
Experience 2–5 yrs
Location Remote (India)
Arrangement Remote
Employment Full-time
Department AI
Responsibilities
- Improve retrieval quality and answer grounding across our content corpus
- Build evaluation harnesses so answer quality is measured, not argued about
- Cut hallucination rates and make refusals honest rather than evasive
- Keep inference fast and cost-sane at student scale
- Work with Content to turn subject expertise into better grounding data
- 2+ years shipping ML or LLM features to real users
- Hands-on with embeddings, vector stores and retrieval pipelines
- Rigour about evaluation — you measure before and after, always
- Strong Python; comfortable in a TypeScript codebase
Nice to have, not required — apply even if none of these are true of you.
- Experience with Upstash Vector, pgvector or similar
- Background in education or assessment
- Published work, open-source contributions or writing on RAG
How we'll judge this role together — shared with you on day one, not at review time.
- Measurable reduction in ungrounded answers against a held-out eval set
- Median answer latency stays under two seconds
- Cost per assisted question trends down quarter over quarter
- 1
Stand up the evaluation harness and establish the quality baseline.
- 2
Own the full AI surface — retrieval, grounding and cost.
- 3
Define AI strategy as the product's intelligence layer deepens.
Skills you'll use
- Python
- RAG
- Vector search
- LLM evaluation
- Prompt engineering
- Remote-first
- Meaningful ownership
- Learning budget
- Full AthenaPrep access
- Equipment that helps
- Time that's genuinely off
Hiring timeline
- 1
We reply within 3 days
A short note on why this problem interests you beats a long CV. Links to things you've made help most.
- 2
30 minutes
A relaxed call about your work, what you want next, and what we're actually building. Questions go both ways.
- 3
Take-home or live
A small, time-boxed piece of work close to what the role actually involves. No trick puzzles, no unpaid project work.
- 4
45 minutes
Values, ownership, and how you'd want the first ninety days to go.
- 5
Within a week
Clear terms, clear scope, and a start date that works for you.
Questions
Yes. We care much more about evidence of craft than years of experience — a project you built and can explain in depth counts for more than a familiar company name.
Yes, remote-first across India, and async by default. We write decisions down so nobody needs to be online at a particular hour to stay in the loop.
Usually under three weeks from application to offer. We reply to every application within three days, even when the answer is no.
Yes, and they are real work with real ownership, not shadowing. Strong interns are regularly offered full-time roles.
Join the talent community below. We open roles as we grow, and the people already in the pool are the first ones we contact.
Think this is you?
A short note on why this problem interests you beats a long CV. If you're close but not an exact match, apply anyway — we'd rather read your note than miss you.
Apply for AI Engineer — Retrieval & Evaluation Join talent community
Apply now