DevOps Engineer (N3+ level)

A fast-growing applied AI company in Japan is seeking a DevOps Engineer to build and deploy LLM-powered applications for large enterprise clients.

This role covers LLM orchestration, RAG data pipelines, backend services, and cloud infrastructure, working closely with frontend developers, product managers, and client engineers to bring AI features into production.

Responsibilities:

-Design and implement prompt and orchestration pipelines using frameworks such as LangChain or LlamaIndex, including prompt chaining, few-shot prompting, and autonomous agents

-Integrate and deploy proprietary and open-source LLMs into production, managing API integrations and keeping models performant and cost-effective

-Build data pipelines for Retrieval-Augmented Generation (RAG) systems, including vector stores and embedding strategies

-Develop and maintain high-performance, fault-tolerant backend services and APIs for AI-centered applications

-Benchmark the performance, latency, and cost of LLM prompts and models through quantitative analysis and A/B testing

-Deploy, manage, and scale applications on cloud platforms such as GCP, AWS, or Azure

-Work with frontend developers and product managers to integrate LLM-powered features into the user experience

-Keep up with the latest LLM, open-source, and cloud technologies, and safely experiment with new solutions

Required Skills:

-Experience with at least one systems language such as Python, Go, Java, C++, or Rust

-Experience with relational and NoSQL databases, and an understanding of vector databases

-Experience setting up, deploying, and operating applications on at least one major cloud provider (AWS, GCP, or Azure)

-Experience building and deploying at least one LLM-powered application (professional work, open-source contribution, or a personal project)

-Familiarity with benchmarking and quantitative analysis methods for AI and application performance

-Strong analytical and debugging skills, and a track record of delivering robust software on time

-A passion for AI and a vision for how it will be used in the future

-Proficient English, and intermediate level of Japanese (Approx. JLPT N3+)

Why Should You Apply?

-Work with some of the sharpest minds in AI in a young, fast-moving company with a flat organization

-Put LLM and AI solutions into production for large enterprises

-Stay hands-on while working closely with client engineers on real-world problems

-Work in an international environment with English as a main working language

-Hybrid work style with flextime

Company Details:

An applied AI company in Japan founded by experienced AI researchers and entrepreneurs. It works with large Japanese enterprises to build AI solutions that improve efficiency, accelerate innovation, and address sustainability challenges.

Work Location: Tokyo

Working Hours: Flextime (core hours 10:00-16:00)

Workstyle: Hybrid

Benefits: Transportation allowance, paid leave, social insurance, periodic health examinations, and skill development and language learning support etc.