DA

Lead Engineer, Compute Infrastructure / Distributed Systems

Hiring from
Japan
Work type
Hybrid
Posted

626,574 remote jobs, straight from company career pages

100% free · New jobs every minute

Show job description

Build a computing system for AI agents. Agent workloads behave differently from the inference loads most GPU infrastructure was built for: context and reasoning tokens grow as a task runs, tool calls arrive in bursts, and many multi-turn sessions run at once. There is no established playbook for running them yet. You will work on problems that have not been solved before, and your answers will run in production.

You will lead the systems side of an unannounced datagusto product: the infrastructure software that runs enterprise AI agent workloads on GPUs, in customers' GPU environments (details under NDA).


What we offer

  • Salary plus stock options
  • Full-time, or start on contract
  • Hybrid in Roppongi, Tokyo; remote at first if based outside Japan


Mission

Take the product from MVP to a production system that runs in customers' GPU environments. Own its systems end to end, and bring the models designed by the Applied Scientist into production.


About datagusto

datagusto Inc. (Roppongi, Tokyo; founded 2020) builds a runtime safety and governance platform that lets enterprises run their AI agents safely.


Why this role exists

datagusto is extending its platform from agent safety and governance into the infrastructure layer where agent workloads run, and this role builds the systems side of it.

As enterprises move AI agents from pilots into production, how those agents run on GPU infrastructure starts to matter, for both cost and performance. datagusto has built its business around understanding what enterprise AI agents are doing at runtime, which gives this product a starting point that infrastructure-only vendors don't have.

This role makes the product actually run on the GPU infrastructure customers already use.


Team

Two people build the product, and the CTO leads the overall effort. This role owns the systems; the Applied Scientist owns modeling and evaluation design.

  • CTO (overall lead): Accountable for the product's overall performance and the PoC numbers; final say on design decisions; weekly syncs with both team members
  • Lead Software Engineer, AI Infrastructure (this role; systems): Core implementation, instrumentation, integration with existing systems, production implementation of the Applied Scientist's models, deployment in customer environments
  • Applied Scientist (1–2 days/week; modeling and evaluation): Designing and prototyping the models the product relies on; designing how they are evaluated; logging specifications
  • CEO (customers): Coordinating with PoC partners; reporting and explaining results


First 3–6 months

  • Run the MVP on real GPU clusters
  • Deploy to a PoC partner's cluster and measure against target metrics
  • Turn it into a production system


Responsibilities

You'll own the product's systems end to end, from core implementation to instrumentation and deployment in customer environments.

  • Implement the product's core logic, and integrate it with the systems customers already run for GPU clusters and LLM serving (the Kubernetes ecosystem and open-source inference stacks)
  • Profile and benchmark LLM inference performance across models, configurations, and load conditions
  • Instrumentation and logging infrastructure across the GPU, inference server, and application layers (DCGM, vLLM and SGLang metrics, application-side events)
  • Production implementation of the models and evaluation pipeline designed by the Applied Scientist, including ways to validate changes safely before rollout
  • Deployment in customer environments, including on-premises and closed networks, and joint validation with customers' engineers
  • Present design options and their rationale in weekly syncs with the CTO


Requirements

You have done at least one of the following in production:

  • Operated LLM inference workloads on GPUs (on Kubernetes, Slurm, or similar) and modified them yourself
  • Led the design of a job scheduler or resource manager for a distributed computing system, and implemented it yourself (e.g., a custom Kubernetes scheduler, Slurm, YARN, Ray, or an in-house equivalent)
  • Led the optimization of a large distributed computing system for throughput, latency, or resource utilization: you made the design decisions and measured the result yourself

If you come from distributed computing, you don't need prior GPU or LLM experience.


You have also done all of the following:

  • Built and operated distributed systems in production, and worked through concurrency, networking, database, queue, and performance problems yourself
  • Completed a master's degree or PhD, or developed software professionally for roughly 5+ years
  • Designed and implemented production systems yourself in Python or Rust (Go is also fine), including debugging, performance analysis, and test design
  • Used coding agents heavily in real development work without taking their output at face value: you worked out the rationale from the high-level design decisions yourself, and spent your review time understanding the critical parts
  • Delivered in a small team where priorities changed often
  • Read and written technical material in English as part of your work (e.g., design docs, papers, issues and pull requests)


Selection

Interview, technical interview, paid work assignment, final interview.

Apply for this job100% free