- Staff / Principal Machine Learning Engineer, Serving
- Remote (US/UK/Canada/Germany/Switzerland/Serbia)
- $270K - $500K + competitive equity
Client provides the world's best realtime AI models for consumer-scale applications. Its core product suite includes:
- Realtime Voice Models — the world's #1 ranked text-to-speech (TTS) and speech-to-text (STT) models, engineered for low-latency, production-grade voice experiences
- Speech-to-Speech API — a Realtime API enabling seamless, natural conversational AI interactions at scale
Who We're Looking For
we're looking for strong people from varied backgrounds who learn fast, thrive in ambiguity, and can show us what they've built, broken, and understood.
Experience We Find Useful
You don't need all of this. But you need enough to make a case.
Inference Optimization. Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.
Model Acceleration. Hands-on experience with quantization, distillation, caching strategies , continuous batching, paged attention, and speculative decoding.
High-Performance Systems. Proficiency in C++, CUDA, Rust, or highly optimized Python. Y
Distributed Systems & Scaling.
Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections.
You can take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production.
Background. PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.
- H-1B transfers supported. Relocation assistance available for candidates willing to move to the SF Bay Area.
- Remote work policy
Tech stack
vLLM, TRT-LLM, CUDA, C++, Rust, Python, Kubernetes, Ray, NVIDIA GPUs, Docker