Agentic Delivery Systems Engineer

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help North American clients innovate faster and more efficiently. As we continue to grow, we’re looking for exceptional people to join our team and help drive impactful change across industries. Why Join AspenView? At AspenView, we’re more than a nearshore IT partner—we’re a people-first, purpose-driven company that believes great culture drives great outcomes. We’re passionate about connecting talent and technology to deliver measurable value for clients—and meaningful career paths for our people. Here’s what you can expect: Competitive base Comprehensive benefits and wellness support Flexible work model: hybrid, remote, or in-office Real growth opportunities and leadership visibility Inclusive, respectful culture that blends U.S. innovation with Colombian heart A company that listens, invests in you, and celebrates wins together About the Role AspenView delivery teams build software with agents in the loop. This role owns how well that actually works. You will design, measure, and continuously tune the agentic delivery system itself: the context that reaches the model, the token and cost envelope it runs in, the tools and permissions it is given, and the evidence that any of it is making teams faster without making the code worse. You will work across accounts, turn what works into reusable configuration and playbooks, and give delivery leadership defensible numbers on throughput, quality, and unit cost. This is a full-time position based in Latin America, working nearshore with our delivery teams and client stakeholders, and it requires English at a C1 level. Core focus areas include context engineering, token and cost optimization, agent and tool configuration, MCP server design, evaluation harnesses, agent observability, delivery metrics and FinOps, CI/CD integration, and hands-on development in Python and TypeScript. What You Will Do Context Engineering & Token Economics Design what enters the context window and what does not: retrieval strategy, file and repository scoping, structured summaries, and pruning rules that keep long sessions coherent. Diagnose and fix context degradation, including window saturation, stale or contradictory context, lost-in-the-middle recall loss, and compaction that silently drops critical state. Optimize prompt caching, session compaction, and reuse to cut redundant token spend without reducing task success rates. Build and maintain cost models for agentic work: cost per task, per merged pull request, and per delivery sprint, with forecasts leadership can plan against. Right-size model selection per task tier, routing routine work to smaller models and reserving frontier capacity for genuinely hard reasoning. Agent Configuration & Harness Design Own system prompts, agent instructions, and repository-level configuration files as versioned, reviewed artifacts rather than ad hoc text passed between engineers. Design the tool surface each agent gets: tool inventories, MCP servers, custom skills, subagent decomposition, and permission and sandboxing models. Build internal MCP servers and integrations that connect agents to the systems delivery teams actually use, including repositories, ticketing, CI, documentation, and data platforms. Maintain a golden reference configuration set with a clear versioning, rollout, and rollback process across client accounts. Evaluation, Benchmarking & Observability Stand up evaluation harnesses with golden task sets that reflect real delivery work, so configuration changes are proven rather than assumed. Run controlled comparisons across models, tools, and prompt strategies, accounting for run-to-run variance before declaring a winner. Instrument agent runs end to end: traces, token and latency telemetry, tool-call success rates, and cost attribution by team, account, and workflow. Maintain a failure taxonomy covering loops, hallucinated APIs, tool misuse, and premature completion, and drive each recurring class to a systemic fix. Own the agentic metrics program end to end: establish pre-agent baselines, publish dashboards, and defend the numbers to delivery leadership and clients. Delivery Workflow Integration Embed agents into the SDLC where they earn their keep: ticket intake and refinement, scaffolding, test generation, code review augmentation, migration and refactor work, and documentation. Define the human-in-the-loop gates — what an agent may merge unattended, what requires review, and what it must never touch. Codify agentic practice into repeatable delivery playbooks and standards, so results hold across engagements rather than depending on which individual engineer is on the team. Partner with delivery leads and client stakeholders to set realistic expectations and report honest results, including where agents underperform. Security, Governance & Enablement Harden agentic workflows against prompt injection, untrusted tool output, secret leakage, and unsafe autonomous actions. Establish governance for enterprise and regulated clients: audit trails, data residency, model and vendor policy, and provenance of generated code. Run enablement for engineers through playbooks, office hours, and internal documentation, and track adoption and proficiency rather than license counts. Act as the internal authority on a fast-moving field: evaluate new tooling, separate signal from marketing, and brief leadership on what genuinely changes the plan. Agentic Metrics You Will Own Baselines are established before agents are introduced, and every metric is reported against that baseline. The example targets below are illustrative starting points; they are calibrated per account and per codebase. Delivery Throughput Cycle time, ticket to merge — whether agents actually compress delivery, not just typing speed. Example target: a 25–40% reduction versus baseline. Review iterations per pull request — hidden cost shifted onto reviewers when agent output is weak. Example target: no increase versus baseline. Output Quality Change failure rate — the share of agent-assisted changes causing a rollback, hotfix, or incident. Example target: at or below team baseline. Code churn within 21 days — code rewritten shortly after merge, the clearest signal of low-quality output. Example target: under 15% of agent-authored lines. Defect escape rate — bugs reaching production per unit of agent-assisted change. Example target: flat or improving. Agent Effectiveness Eval pass rate (pass@1) — task success on the golden set, and the gate for any configuration change. Example target: no regression, trending up. Human intervention rate — the share of sessions needing correction, redirection, or abandonment. Example target: under 30% and falling. Tool-call error rate — malformed or failed tool invocations, usually a configuration defect. Example target: under 3% of calls. Context Health Context utilization at completion — how close tasks run to window saturation before finishing. Example target: under 60% of the window. Prompt cache hit rate — how much repeat context is being paid for twice. Example target: above 70% on long sessions. Retrieval precision — the share of loaded context the model actually uses, measuring wasted tokens. Example target: above 50%. Unit Economics Cost per merged pull request — the headline number for pricing engagements and proving value. Example target: tracked and trending down. Tokens per completed task — split across input, output, and cached to show where spend concentrates. Example target: down quarter over quarter. Model mix ratio — the share of work routed to smaller models without hurting pass rate. Example target: above 50% on routine tasks. Adoption & Safety Active usage depth — sessions per engineer per week across workflow types, not seat counts. Example target: breadth over raw volume. Blocked unsafe actions — guardrails catching secret exposure, unsafe writes, or injected instructions. Example target: 100% logged and reviewed. What Success Looks Like in the First Year Cost per merged pull request trends down while change failure rate holds flat or improves. An evaluation suite covers the top delivery workflows and gates every configuration change. Golden agent configurations are versioned, documented, and adopted across multiple accounts. Delivery leadership can answer, with evidence, what agentic delivery is producing per dollar. What You Bring Education Bachelor’s degree in Computer Science, Software Engineering, or a related technical field, or equivalent hands-on professional experience. Experience 5+ years building and shipping production software, with real depth in at least one ecosystem. Hands-on production experience with agentic coding tools such as Claude Code, Cursor, GitHub Copilot, Codex, or Aider, beyond casual use. Demonstrated work on context engineering, agent optimization, or LLM evaluation, with measured outcomes you can talk through. Measurement discipline: you instrument first, form a hypothesis, and let the data overrule your intuition. Technical Expertise Strong Python and/or TypeScript, and comfort building tooling, harnesses, and integrations. Working knowledge of LLM mechanics: context windows, tokenization, caching, sampling, tool calling, and where each one breaks. Solid CI/CD, version control, and testing fundamentals — this is a delivery system, not a chatbot. Microsoft Azure DevOps (required): hands-on with Boards, Repos, and Pipelines, and able to integrate agentic workflows into that toolchain. Language Skills English at C1 level, with the ability to explain trade-offs to engineers and executives alike. Nice to Have Hands-on production experience with OpenAI Codex, including configuration and workflow integration (preferred). MCP server development or contributions to open-source agentic tooling. Experience in regulated environments such as financial services, healthcare, or automotive, where governance is non-negotiable. Background in developer experience, platform engineering, or site reliability engineering. Familiarity with evaluation frameworks, LLM observability platforms, or DORA-style delivery metrics. Visa Sponsorship AspenView does not provide visa sponsorship for this role. Candidates must already be legally authorized to work in their country of residence. Equal Opportunity Employer AspenView is proud to be an equal opportunity employer. We believe in creating an environment where all employees feel welcome, valued, and empowered to succeed. We celebrate diversity and strive to build a culture of inclusion where all individuals, regardless of their race, color, gender, gender identity or expression, sexual orientation, disability, age, or any other characteristic, can thrive. We encourage applicants from all walks of life to join our team and make a lasting impact. Visa Sponsorship AspenView does not sponsor employment visas for this role. Applicants must be currently authorized to work in the United States on a permanent basis without the need for visa sponsorship now or in the future. Equal Opportunity Employer AspenView is proud to be an equal opportunity employer. We believe in creating an environment where all employees feel welcome, valued, and empowered to succeed. We celebrate diversity and strive to build a culture of inclusion where all individuals, regardless of their race, color, gender, gender identity or expression, sexual orientation, disability, age, or any other characteristic, can thrive. We encourage applicants from all walks of life to join our team and make a lasting impact.