PayFit logo

Senior AI Engineer (Quality & Knowledge)

Hiring from
France
Work type
Hybrid
Posted

624,714 remote jobs, straight from company career pages

100% free · New jobs every minute

Show job description

Get to know us


Do you want to help us shape what the future of work will look like and how it will best embrace our life's aspirations? If this sounds like a journey you want to embark on, we may have the right role for you! PayFit is an intuitive software as a service payroll and HRIS solution designed specifically for SMBs. Since 2015, we have set ourselves a mission to simplify payroll for SMBs and enable employers and employees to grow together. We are a European company operating from 3 main countries (France, Spain, and the UK) where we are supporting over 20,000 clients.

Creating a fulfilling work environment and culture is also a core mission at PayFit, and our day-to-day work philosophy is reflected in our four values:

Care: We genuinely care for others whoever they are, whatever they think.

Excellence: We aim to improve and achieve better results every day.

Humility: Staying humble and exchanging transparent feedback helps us to grow and improve.

Passion: We are the architects of PayFit's success.

A key part of our culture, and essential for our success, is also improving the diversity of our teams and building an inclusive culture where you can be yourself at work.

This is why our recruitment focuses on the skills you demonstrate, and not only on your academic background or previous professional experiences. At PayFit we understand that you can gain applicable skills through a variety of life experiences and we are interested in knowing them, too.

Location :
Flexible location: work remotely from anywhere in France or Spain

The team

You'll join the team behind Payfit AI, building AI agents that help companies prepare, check, and run payroll, so people can focus on the decisions that need their expertise.

We work closely with Customer Support and payroll experts to understand real customer situations and turn that expertise into useful product experiences. These span a wide range of payroll and HR topics, from everyday questions to complex cases in a highly regulated environment.

You'll contribute to connecting that expertise with the systems we build. Using real cases, you'll help evaluate the whole workflow: did the agent give the right answer, take the right action, or know when to ask for clarification or involve a human? Where a workflow affects payroll, evaluating its quality also means checking the resulting outcome.

This is a hands-on senior engineering role. You'll build on existing models, agents, evaluation tools, and knowledge pipelines. Depending on what you find, you might improve the knowledge base, create a specialized agent, connect a new tool, or evolve the agent harness: how agents receive context, use tools, apply guardrails, and hand off to a person. You'll use evaluations and production feedback to guide these improvements, measure their impact, and catch regressions as models, payroll rules, and workflows evolve.

What you'll do
  • Work with Customer Support and payroll experts to understand how they resolve complex cases and turn that expertise into testable AI behavior.

  • Build and improve evaluations using representative cases, expert reviews, and production feedback to assess answers, actions, and workflow outcomes.

  • Investigate failures across models, context, knowledge, retrieval, and tools, then implement and measure improvements.

  • Contribute to specialized agents, orchestration, and harness engineering, including context management, reusable tools, guardrails, and human handoffs.

  • Improve knowledge pipelines and retrieval quality, and use regression evaluations and observability to keep workflows reliable as they evolve.

What we're looking for:
  • 7+ years of software engineering experience, including designing and operating backend services or pipelines in production.

  • Hands-on experience building LLM applications or agents, including tool integration, context management, and handling uncertainty.

  • Practical experience evaluating LLM systems using representative datasets, clear criteria, and automated or human review—and using the results to guide improvements.

  • Strong backend engineering skills and the ability to contribute to a TypeScript/Node.js codebase.

  • Comfort working directly with domain experts, translating ambiguous cases into testable behavior, and collaborating on reliable production solutions.

  • Care for security, permissions, observability, and maintainability.

Nice to have
  • Experience with Langfuse or similar tools for tracing, prompt management, datasets, and experiments.

  • Familiarity with AI SDK or comparable frameworks for building tool-using agents.

  • Experience with hybrid search, reranking, OpenSearch, or other search and vector technologies.

  • Experience measuring judge agreement, evaluation uncertainty, or agent consistency across repeated runs.

  • Experience building annotation tools or review workflows for domain experts.

  • Familiarity with AWS, data pipelines, and CI/CD for AI systems.

  • Experience in payroll, HR, customer support, or another domain where factual accuracy and traceability matter.

Our hiring process
  • Step 1 — 30-45 min call with Talent Acquisition

  • Step 2 — 45-minute meeting with our Hiring Engineering Manager

  • Step 3 — 60-minute technical interview with engineers from the squad

  • Step 4 — 30-minute meeting with our Product Manager and AE Designer/CS expert

  • Step 5 — 45-minute final interview with the Tribe Engineering Director
    Background Check (ISO27001 certification): identity, diploma, and past professional experience will be checked after the offer.

What we offer

Flexibility
: We believe it is key to producing your best work and being fulfilled. We therefore offer the possibility to work away from our main offices, within France, as well as abroad for a defined period. Further requirements may apply depending on the role and your overall experience.

Learning & Development:
At PayFit, we offer a comprehensive learning platform that enables you to acquire new skills daily, supported by our company.

Career Development:
We want you to progress and be free to choose which direction you want to grow. There are also opportunities for internal moves.

Health insurance: Henner Mutuelle Insurance (60% covered by PayFit, free coverage for children)

Transportation: 50% of public transportation costs are covered by PayFit for those living within the Ile de France region, or assistance with sustainable mobility (a bicycle rental subscription, purchase of soft mobility equipment to get to work, etc.).

Meals: A restaurant card with our partner Swile (9€ per workday) covered at 60% by PayFit

A Work Council grant: A monthly allowance to be spent on culture, sports, personal services, etc., as well as a vacation bonus.

Work equipment:
A MacBook is our standard working tool


Parental support program:
Salary maintenance during the first month of additional parental leave.


Time off:
25 days of holidays + RTT days (depending on the contract).

Disability Inclusion:
All of our positions are open to any person living with a disability. To guarantee equal treatment and opportunities, we will take, based on individual needs, appropriate measures to adapt the work conditions of PayFiters with disabilities, and if needed also during the recruitment process. Please let us know what you need and we will do our best to accommodate!

Similar jobs

Apply for this job100% free