Senior Machine Learning Engineer (Inference Platform)

About Wizard AI At Wizard AI, we’re building a high-performing AI Shopping Agent that helps people discover the best products across the web with speed, accuracy, and trust. Our ML systems sit at the core of that experience, and we’re looking for a Senior MLOps Engineer to help us run them reliably and efficiently in production. The Role As a Senior MLOps Engineer at Wizard, you’ll own the end-to-end lifecycle of our ML systems — from packaging and deployment to monitoring, performance, and scaling — across a custom-built inference platform powering a live conversational product. This isn’t a typical “pipeline” role. Our platform runs multiple specialized inference engines (LLMs, embeddings, and extraction models), each with different performance and scaling characteristics. A big part of the role is thinking through tradeoffs — latency vs. cost, throughput vs. reliability — and helping us evolve the system as we grow. You’ll work closely with ML, Data, and DevOps, and have real input into how the platform is designed — not just how it’s maintained. What You’ll Do Build and improve production ML pipelines, making it easy to move models from experimentation to reliable production use Help own and evolve our multi-engine inference platform (LLMs, embeddings, and extraction), improving how different workloads are served and scaled Put strong foundations in place for model versioning, rollouts, and rollbacks so systems stay reproducible and safe to iterate on Define and monitor key system metrics like latency, availability, and GPU utilization, and set clear expectations around performance Improve overall system performance — whether that’s reducing latency, increasing throughput, or making better use of GPU resources Design systems that are resilient and cost-aware, with thoughtful approaches to autoscaling, failure isolation, and graceful degradation Bring solid engineering practices (testing, CI/CD, observability) into ML workflows to help the team move faster without sacrificing reliability Partner closely with ML, Data, Product, and DevOps to turn ideas into production-ready systems and help guide technical decisions What We’re Looking For 5–8+ years of experience in software, ML, platform, or infrastructure engineering, with hands-on ownership of production ML systems Experience deploying and running LLMs or other deep learning models in real-world environments Strong Python skills and a solid foundation in software engineering Familiarity with cloud platforms (AWS, GCP, Azure) and common ML tooling (model registries, experiment tracking, etc.) A good understanding of inference performance — batching, memory usage, quantization, and how systems behave across CPU and GPU Experience working with (or curiosity about) systems that serve different types of models with different constraints Ability to think through tradeoffs between speed, cost, and reliability in a practical way Comfort working in a fast-moving environment where things evolve quickly What Success Looks Like Reliable, Scalable Systems Our ML systems run smoothly with clear visibility into performance, and can scale as demand grows without constant firefighting. End-to-End Ownership You’re able to take a model from idea to production and keep it running well, while making it easier for others to do the same. Real Impact You help shape how our ML platform evolves — improving performance, reducing costs, and making the overall system stronger over time. Compensation & Benefits The expected base salary range for this role is $200,000 – $250,000 USD, and will vary based on skills, experience, role level, and geographic location. Final compensation will be determined by considering these factors alongside overall role scope and responsibilities. In addition to base salary, Wizard offers: Equity in the form of stock options Medical, dental, and vision coverage 401(k) plan Flexible PTO and company holidays Fully remote work within the United States Periodic company offsites and team gatherings Wizard is committed to fair, transparent, and competitive compensation practices.