Lead DevOps/MLOps Engineer

We are looking for a Lead DevOps/MLOps Engineer to join our ML Engineering team and help build out and harden our on-premises, GPU-backed ML platform.

The goal is to enable multiple ML and LLM use cases to run simultaneously, reliably and with low latency, while scaling the platform to a wider user base. You will work within a 25-person Data Science Department that has grown an in-house agentic platform from a small pilot to roughly 500 employees, with further rollout currently limited by compute capacity. Since all solutions handle confidential documents and sensitive data, the platform runs fully on-premises across 12 (and growing) NVIDIA H200 GPU servers, two Kubernetes clusters, and a stack that includes MLflow, Ray, and an Agent Gateway.

Responsibilities

• Build out and harden the on-premise, GPU-backed ML platform to support multiple concurrent ML and LLM workloads
• Ensure reliability, low latency, and scalability as the user base expands beyond the current ~500 employees
• Mature the GPU environment to production-grade standards, addressing daily bottlenecks and resource contention
• Manage and evolve the GPU infrastructure across 12+ NVIDIA H200 servers, including integration with Kubernetes where appropriate
• Operate and improve the two Kubernetes clusters (production and sandbox) along with platform services such as MLflow and the Agent Gateway
• Support self-hosted LLM deployments for internal, confidential data use cases
• Maintain and extend the Ray environment shared by the Data Science and Deep Learning teams for training workloads
• Enhance monitoring and observability using NVIDIA DCGM exporter, Prometheus, and Grafana
• Collaborate closely with the Traditional ML, Deep Learning, and Data Analytics teams to align infrastructure with evolving use cases

Requirements

• 5+ years of experience in DevOps/MLOps engineering roles
• At least 1 year of relevant leadership experience
• Hands-on experience running GPU workloads on Kubernetes or OpenShift in production, ideally on-premises
• Practical experience deploying and troubleshooting the NVIDIA GPU Operator with a clear grasp of what each component does
• Understanding of how GPUs differ from CPUs as a scheduled resource, including memory limits, utilisation semantics, and the limitations of default Kubernetes scheduling
• Knowledge of GPU sharing and isolation methods such as MIG, time-slicing and MPS, and their trade-offs for memory and fault isolation
• Solid Linux fundamentals including cgroups, namespaces, and systemd resource controls for process-level resource management
• Skills in Prometheus, Grafana and NVIDIA DCGM, including capturing GPU metrics at process or pod level
• Working knowledge of the CUDA stack covering drivers, CUDA toolkit, container runtime, and version compatibility
• Experience deploying with Helm and operating MLOps tools such as MLflow and Airflow on Kubernetes
• English proficiency at an Upper-Intermediate level (B2) or higher

Nice to have

• Familiarity with LLM serving engines such as vLLM, SGLang or NVIDIA Triton
• Experience with Ray/KubeRay and GPU-aware schedulers such as Kueue or Volcano
• Hands-on work with H100/H200-class hardware, GitLab CI and Docker
• Background in Machine Learning and Python

We offer

CONTINUOUS UPSKILLING, LEARNING & DEVELOPMENT

• Diversity of tasks and projects
• Assessment center for objective review of competency level
• Personal development plan
• Mentoring programs and leadership development
• Certification and professional development support 
• Access to learning platforms including more than 2,500 internal courses
• English courses taught by certified teachers

CORPORATE BENEFITS

• Extra leave days
• Referral bonuses

COMPENSATION PACKAGE

• Competitive compensation paid in USD
• Regular salary and performance reviews

MEDICAL & HEALTHCARE

• Private health insurance
• Well-being events

WORKING ENVIRONMENT

• Recreation areas and kitchens
• Tea, coffee and snacks
• Sports equipment and game consoles
• IT Equipment
• Microsoft’s Software Assurance Home Use Program (HUP)

Please note that our Talent Attraction Team reviews applications and CVs submitted in English.

EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.

Remote in Türkiye