Senior ML / AI Operations Engineer

Role Overview

We are seeking a Senior ML / AI Operations Engineer to architect, deploy, and scale our core AI compute platform. Operating at the unique intersection of large scale machine learning and AI and critical energy infrastructure, our team builds the intelligent platform that powers renewable energy interconnection, autonomous grid dispatch and optimization, -long-term grid planning, and energy-domain foundation models.

In this role, you will bridge the gap between applied research and industrial grade production systems. You will be responsible for orchestrating distributed GPU/TPU training clusters, high-throughput inference serving pipelines, and the agentic execution environments for state-of-the-art agent workflows. You will ensure that our intelligent energy models deliver high reliability, low latency, and operational excellence at petabyte scale.

Key Responsibilities:

Distributed Training Infrastructure:

• Architect, build, and maintain multi-node, multi-GPU distributed training compute clusters (Kubernetes, Slurm, Ray) optimized for multi-billion parameter foundation models and massive spatiotemporal time-series architectures.
• Optimize resource allocation, job scheduling, and fault tolerance across heterogeneous hardware fleets to ensure maximum compute efficiency.
• Implement fault-tolerant checkpointing, automated distributed state recovery, and distributed data pipelines capable of ingesting gigabytes per second of streaming telemetry.

Inference Serving & High-Throughput Pipelines

• Design and operate mission-critical, low-latency model serving microservices handling real-time grid orchestration and continuous load forecasting.
• Deploy optimized inference runtimes (Triton Inference Server, TensorRT, vLLM, TGI) across cloud regions and localized edge compute nodes deployed at substation sites.
• Establish scalable training and Human-In-The-Loop deployment pipelines integrated with feature stores, dynamic model routing, and shadow/canary release strategies.

Modern LLMOps & GenAI Ecosystem

• Implement end-to-end LLMOps infrastructure for fine-tuning (LoRA, QLoRA, DeepSpeed, Megatron-LM), alignment (RLHF, DPO), and evaluating domain-specific foundation models.
• Build high-performance Retrieval-Augmented Generation (RAG) platforms utilizing distributed vector databases (Milvus, Qdrant, Pinecone) integrated with real-time energy domain knowledge bases.
• Implement runtime guardrails, prompt caching, token cost governance, continuous evaluation, and hallucination monitoring for production GenAI applications.

Observability, Reliability & Governance

• Deploy holistic MLOps observability stacks covering GPU/NPU hardware metrics, inference latency distributions, data drift, concept drift, and model performance decay.
• Enforce rigorous security, data governance, and compliance standards for sensitive infrastructure operational data.
• Lead incident management, root-cause analyses (RCA), and continuous reliability improvements across all production AI services.

Technical Qualifications:

Core Requirements:

• Industry Experience: 5+ years of software engineering or DevOps experience, with 3+ years dedicated to MLOps, LLMOps, or AI platform infrastructure operating at scale.
• Orchestration & Containers: Deep technical mastery of Docker, Kubernetes, Helm, and custom Kubernetes Operators for workload scheduling and cluster management.
• ML Frameworks & Libraries: Proficiency with PyTorch, Ray, vLLM, Hugging Face DeepSpeed, and Triton Inference Server.
• Infrastructure as Code & Cloud: Hands-on experience with Terraform, Ansible, and enterprise cloud ecosystems (AWS, GCP, or Azure).
• Programming Languages: Advanced proficiency in Python and C++ or Rust for high-performance system engineering.
• Data Pipelines: Solid understanding of streaming and batch data architectures using Kafka, Spark, Flink, or Apache Iceberg.

Preferred Qualifications:

• Experience with machine learning applied to the energy domain, especially dealing with datasets spanning imagery, time series, grid models and text.
• Direct experience deploying edge AI models to constrained industrial hardware using TensorRT, ONNX Runtime, or OpenVINO.
• Track record of contributions to open-source MLOps or LLMOps frameworks (Ray, vLLM, Kubeflow, LangChain, LlamaIndex).

Our values

• Take charge: We take initiative and own outcomes that move the mission forward.
• Transform with purpose: We build solutions that solve real problems and create meaningful impact.
• Be a Tapestry, not a thread: We collaborate across diverse skills and perspectives to achieve more than we can individually.
• Always fine-tune: We stay curious, seek feedback, and refine our understanding as we learn.
• Stay grounded: We listen openly, value different perspectives, and stay focused on what matters most.

What we offer

A culture that supports growth, ownership, and meaningful impact, along with:

• Competitive salary and equity
• Medical, dental, and vision coverage
• Generous PTO and flexible hybrid work model
• 401(k) with employer contribution
• Professional development
• The ability to work on important real-world problems within an Alphabet-backed environment

The US base salary range for this full-time position is $166,000 - $244,000 + bonus + equity + benefits. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your location during the hiring process.

Please note that the compensation details listed in US role postings reflect the base salary only, and do not include bonus, equity, or benefits.

.