Engineering Manager, Evaluations & Observability

ABOUT RETOOL Nearly every company in the world runs on custom software: Gartner estimates that up to 50% of all code is written for internal use. This is the operational software for refunding orders, underwriting loans, onboarding employees, analyzing transactions, and providing customer support. But most companies don’t have adequate resources to properly invest in these tools, leading to a lot of old and clunky internal software or, even worse, users still stuck in manual and spreadsheet flows. At Retool, we’re on a mission to bring good software to everyone. We’re building a new type of development platform that combines the benefits of traditional software development with a drag-and-drop UI editor and AI, making it dramatically faster to build internal tools. We believe that the future of software development lies in abstracting away the tedious and repetitive tasks developers waste time on, while creating reusable components that act as a force multiplier for future developers and projects. The result is not just productivity, but good software by default. And that’s a mission worth striving for. Today, our customers span from small startups building their first operational tools to Fortune 500 companies building mission-critical apps for thousands of users across their business. Interested in joining us? Let us know! WHY WE’RE LOOKING FOR YOU: Our engineering leaders are at the forefront of Retool’s product development, bridging the gap between engineering excellence and customer impact. We look for leaders who not only bring strong technical expertise but also the strategic vision to shape Retool’s product direction, balancing day-to-day execution with long-term thinking. In this role, you’ll lead Retool’s Evaluations & Observability platform. You’ll own setting the bar for what “good” looks like across our newly-launched Assist experience, making sure it works and works consistently , at scale. You’ll build the systems, tools, and culture that let us measure, understand, and improve quality in real time, driving relentless iteration across everything we ship. You’ll guide engineers focused on: Evaluation platforms: building the frameworks that let us test and compare performance across LLM providers and model versions Quality systems: defining and enforcing rubrics, metrics, and evaluation loops that answer the hardest question in AI: “Is this actually good?” Data curation: managing the datasets that power and test our AI models, sourced from real-world usage to keep our systems grounded and relevant Search & retrieval quality: owning the retrieval layer that underpins both AI and non-AI experiences ensuring results are relevant, accurate, and fast Reusable AI quality infrastructure: creating the building blocks (evaluation tools, pipelines, and feedback systems) that other teams can leverage to maintain quality across Retool’s AI surface area Culture of continuous improvement: embed a data-driven approach to AI quality, where experimentation and measurement are the default as we scale our capabilities IN THIS ROLE, YOU WILL: Communicate and collaborate effectively with Product and other Engineering counterparts Manage a team of engineers; support the team by identifying growth opportunities, providing continuous feedback, and performance management as appropriate Understand the needs of our Assist roadmap, helping define rubrics & automated systems that allow engineers to iterate quickly on product features, with confidence Establish and define your team's strategy to ensure execution maximizes business impact Introduce scalable, repeatable processes that help engineering and product teams deliver a successful product Partner with recruiting on building out a diverse team of exceptionally motivated engineers THE SKILLSET YOU'LL BRING: 3+ years of experience successfully leading and managing teams Familiarity with AI evaluation & observability systems – we use Braintrust, but exposure to general evaluation & LLM observability systems is a plus Deep technical curiosity. You’ll be writing code, engaging deeply on design and architecture, and tinkering to uncover what’s really happening under the hood A strong ability to champion an engineering team through macro process changes A history of orienting teams and setting strategy toward engineering goals, with a bias toward impact A propensity to operate as a business-owner—caring deeply about our customers, product, and team Thoughtfulness around engineering culture, process, and identity Proficiency in navigating through ambiguity, managing stakeholders, communicating in a structured manner, as well as driving maximal accountability and excellence A collaborative skillset to partner effectively with product, design, and go-to-market teams For candidates based in the United States, the pay range(s) for this role is listed below and represents base salary range for non-commissionable roles or on-target earnings (OTE) for commissionable roles. This salary range may be inclusive of several career levels at Retool and will be narrowed during the interview process based on a number of factors such as (but not limited to), scope and responsibilities, the candidate’s experience and qualifications, and location. Additional compensation in the form(s) of equity and/or commission are dependent on the position offered. Retool provides a comprehensive benefit plan, including medical, dental, vision, and 401(k). Pay and benefits are subject to change at any time, consistent with the terms of any applicable compensation or benefit plans. The base pay range for this role is $260,100 – $306,000 per year. Retool offers generous benefits to all employees and hybrid work location. For more information, please visit the benefits and perks section of our careers page! Retool is currently set up to employ all roles in the US and specific roles in the UK and Mexico. To find roles that can be employed in the UK and Mexico, please refer to our careers page and review the indicated locations.