Senior ML/Platform Engineer

Senior ML/Platform Engineer, Bailey AI What This Role Is Bailey is a clinical AI system that orchestrates LLMs, manages health safety evaluation, and integrates with clinical data to help people navigate their health. Three data scientists build it. Nobody currently owns what happens after something works. This role owns the path from prototype to production and the infrastructure that tells us whether production is working correctly. You'll build deployment pipelines, evaluation frameworks, monitoring systems, and the observability layer that makes a clinical AI system trustworthy, not just functional. This is not a role where you maintain someone else's platform. You'll design and build it. The team is small enough that your decisions shape the system's architecture directly. Your output is running systems, not notebooks or papers. You need to understand what you're deploying, not just how to deploy it. Much of this work hasn't been defined yet; the evaluation framework exists as a strategy document, the monitoring system exists as a gap, and you turn those into infrastructure. What You'll Actually Do We're looking for a fast ramp here — this role is senior specifically because it operates without a playbook, and that means starting to produce quickly, not spending a quarter getting oriented. In your first 30 days: Deploy and version ML artifacts that currently move to production ad hoc Stand up observability for the existing LangGraph agent orchestration (what's happening, how often, how well) Instrument the health safety evaluator so its 5-dimension scoring is monitored continuously, not checked manually Within 3 months: Build the evaluation infrastructure described in our product strategy: drift detection, distributional monitoring, safety-weighted quality scoring Own CI/CD for ML artifacts and agent configurations Harden existing services (conversation persistence, MCP tool orchestration, clinical skills framework) for production reliability Get up to speed on Bailey's evaluation documentation and use it to shape how we test and monitor Bailey's behavior going forward, including flagging where traditional ML methods (not just LLM-based approaches) could deliver the same outcome more efficiently Ongoing: Be the person who knows whether the system is healthy, degrading, or broken before users or clinicians notice Collaborate with data scientists so their work reaches production without a handoff gap Keep testing and monitoring aligned with Bailey's evolving evaluation documentation, and keep surfacing opportunities to swap in traditional ML where it's more efficient than an LLM-based approach Evolve the platform as the product grows (voice, memory, multimodal are on the roadmap) What You Bring These are real requirements, the things that would be hard to learn on the job at the pace this role demands: You've built production ML infrastructure. Not just used it. Deployment pipelines, serving systems, model tracking/registries (MLflow or equivalent), monitoring that pages you at 2am. You know the difference between a model that works in a notebook and one that works in production. You're a strong software engineer first. Distributed systems, containerization, event-driven architectures, CI/CD. ML is the domain; engineering is the craft. You're comfortable owning something end-to-end. No one will hand you a spec and check your work weekly. You'll identify what needs to exist, propose how to build it, and ship it. You work well in small teams. You communicate clearly, you surface problems early, and you're comfortable with the visibility and accountability that comes with a four-person team. What Would Make You Particularly Effective Here These aren't requirements. They're things that would shorten your ramp or deepen your impact: Experience in healthcare or regulated industries (you understand why audit trails and compliance aren't afterthoughts) Background in ML research or applied ML (you can have informed opinions about model behavior, not just model infrastructure) Familiarity with AWS ML services like Bedrock and SageMaker, which are part of our current stack Experience with LLM orchestration frameworks (LangChain, LangGraph, or similar) Exposure to clinical data standards (FHIR, HL7) or health information systems Publications, open-source contributions, or side projects that show you go deeper than your day job requires About the Team You'll join a small data science team that builds Bailey's AI capabilities: models, integrations, clinical skills, and orchestration. The team works closely with engineering and product. You'll be the first person dedicated to production infrastructure and evaluation systems, which means you'll shape how those concerns are handled from the ground up. The target salary range for this position is $150,000 - $190,000 annually and is part of a competitive total rewards package including stock options, benefits, and incentive pay for eligible roles. Individual pay may vary from the target range and is determined by a number of factors including experience, location, internal pay equity, and other relevant business considerations. We review all employee pay and compensation programs annually at minimum to ensure competitive and fair pay. Data shows that women, people of color, and other underrepresented groups may be less likely to apply for jobs unless they believe they are a perfect match. But b.well holds diversity amongst its key values, and we have a strong commitment to building our workforce and products through that lens. You don't have to check every box in this job description to be a great fit for the role! If you're excited about this position and the prospect of working for b.well, please apply. If it turns out this role isn't for you, there may be other openings that could align with your experience and expertise! We are committed to an inclusive and diverse b.well. We are an equal opportunity employer. We do not discriminate based on race, ethnicity, color, ancestry, national origin, religion, sex, sexual orientation, gender identity, age, disability, veteran, genetic information, marital status or any other legally protected status