Applied AI Engineer

About mpathic

mpathic is building the future of trustworthy AI. Grounded in behavioral science and human-centered design, we provide the infrastructure for building AI systems that are safe, aligned, and emotionally intelligent.

Building on our work in areas like RL gyms, red teaming, and benchmarking, we are creating the foundation for training, probing, and measuring advanced AI systems reliably, auditably, and at scale.

We serve frontier AI labs, large enterprises building and deploying AI, and Life Sciences organizations. Our Life Sciences business brings clinician-led AI monitoring, safety detection, and expert clinical review to clinical trials, helping sponsors and CROs protect participants and data quality, with a strong presence in psychedelic and CNS research.

Position Overview

We are looking for an Applied AI Engineer to own how mpathic's production services are built on large language models (LLMs). You will sit on the platform engineering team, reporting to the CTO, and work closely with the AI/ML team, research, and Security and Compliance. The AI/ML team defines how a model should behave: which model, which prompt, which judge, and the evaluation bar it must meet. You make that behavior run safely, reliably, and cost-effectively in production at scale.

Our platform spans red teaming, dataset generation, annotation, and evaluation, much of it for healthcare and life sciences customers where quality and auditability matter. Your job is to build the paved road every team uses to ship LLM-powered features, so that every model and prompt change reaching a customer arrives through the same tested, observable, and auditable path.

This is a hands-on engineering role. You should be as comfortable designing a gateway or a CI release gate as you are sitting with a researcher to turn a working prototype into a production service. You will start as the owner of LLM and inference infrastructure in production, with room to shape how we build with AI as the team grows.

What You'll Accomplish

In your first 60-90 days you'll…

• Build a deep understanding of the platform, its LLM-backed services, and how a model or prompt change moves from prototype to production today.
• Inventory every place production code calls an LLM: which providers, models, features, and data each one uses.
• Publish the first approved list of models, providers, and features, with data retention, BAA coverage, and version pinning documented for each.
• Stand up the first version of the LLM gateway and move at least one production service onto it.
• Build trusted relationships across platform engineering, the AI/ML team, research, and Security and Compliance.

In your first year you'll…

• Route all production LLM traffic through the gateway, with logging, cost attribution, failover, and PHI controls.
• Ship a release path for model and prompt changes: versioning, evaluation gates in CI, staged rollout, and rollback.
• Take AI/ML prototypes, with their eval harnesses and acceptance thresholds, into scalable production services.
• Deliver production monitoring for latency, cost, failures, and quality drift against the evals defined by the AI/ML team.
• Produce the audit evidence for LLM changes that SOC 2, ISO 27001, HIPAA, and FDA 21 CFR Part 11 require.

You'll Thrive in This Role If You…

• Are a backend engineer first, and have carried LLM features through what comes after the demo: evals, regressions, cost, latency, outages, and audits.
• Enjoy building the platform other people build on, and measure success by how easily and safely the rest of the team ships.
• Treat a prompt change with the same rigor as a code change, and care deeply about reliability, reproducibility, and auditability.
• Understand modern AI evaluation concepts (LLM-as-judge, red teaming, regression evals) and how to turn them into automated release gates.
• Are comfortable with high-stakes, sensitive subject matter such as healthcare, safety, and wellbeing, and the data handling it demands.
• Can translate between researchers, clinicians, and engineers, turning a working prototype into a production service without losing what made it work.
• Communicate crisply through decision records, design docs, and diagrams, and can align people around a standard without relying on authority.
• Have shipped real LLM-powered products in production, shown through what you have built rather than years on a résumé.

What You'll Do

Own the LLM Gateway and Inference

• Design, build, and run the single path through which production services call model providers.
• Build in logging, cost attribution, rate limiting, failover, and controls for protected health information (PHI).
• Remove direct provider API keys from services, so nothing reaches a model except through the gateway.

Own the Approved Model Catalog

• Maintain the approved list of models, providers, endpoints, and model features, such as structured outputs, tool use, caching, and batch.
• Document data retention, BAA coverage, and version pinning for each, and record every addition in a short decision record.
• Keep the team current on changes in the model and provider landscape, and on what they mean for our stack.

Build the Release Path for Model & Prompt Changes

• Version prompts and model configurations alongside code.
• Run the AI/ML team's evals as release gates in CI, with staged rollout and rollback.
• Give engineers and researchers self-service templates, so they move fast in sandbox and staging while production stays gated.

Productize AI/ML Prototypes

• Take a validated prototype, its eval harness, and its acceptance thresholds, and turn them into a scalable, observable production service.
• Advise the AI/ML team early on the cost, latency, and availability of the models they choose.

Observe, Operate LLM/Inference Runtime

• Instrument LLM services end to end with OpenTelemetry: latency, failures, token usage, cost, and output quality.
• Monitor for drift against the AI/ML team's evals, join on-call, and lead incident response for LLM-backed services.
• Partner with Security and Compliance to produce audit evidence for SOC 2, ISO 27001, HIPAA, and FDA 21 CFR Part 11.

Experience

Core experience

• 5+ years of backend software engineering, including multi-tenant SaaS services and APIs in production.
• Production experience building on commercial LLM APIs (for example Anthropic, OpenAI, AWS Bedrock, or Azure OpenAI), including prompt management, structured outputs, and tool use.
• Hands-on experience designing evaluation pipelines for LLM output and wiring them into CI/CD as release gates.
• Strong Python; NodeJS is a plus.
• AWS, containers (ECS or similar), and infrastructure as code (Pulumi, Terraform, or CDK).
• Observability tooling such as OpenTelemetry, Datadog, or similar, and the delivery toolchain (version control, feature flags, CI, and an issue tracker such as Linear).
• Working knowledge of application security: secrets management, access control, and handling of sensitive data.

Preferred

• Experience in regulated domains such as healthcare or life sciences, including HIPAA, SOC 2, ISO 27001, or FDA 21 CFR Part 11 validated systems.
• Experience building an internal developer platform or paved road used by other teams.
• Experience with LLM-as-judge evaluation, red teaming, or conversational analytics.
• Experience with training or fine-tuning models from scratch.

About the Team

You will work closely with:

• CTO and platform engineering: to scope, sequence, and ship the work.
• ML Team, Chief Science Officer and Research Team: to turn validated direction and scientific requirements into shippable product.
• Domain experts: to ground the work in real practice and quality of findings.
• Sales and the delivery teams (Post Sales Support and AI Labs Services Delivery): to align what we build with customer needs and deliverables.

We value operational excellence, systems thinking, and high standards for quality, clarity, and auditability.