Member of Technical Staff - Applied ML

About the Role

This is an applied ML engineering role at a Series B fintech company building agentic AI for accounting professionals. You will own end-to-end projects that bring intelligence into production, helping agents reason, plan, and evaluate themselves across complex, real-world accounting workflows. It is a high-autonomy position that blends research-minded thinking with practical engineering.

What You'll Do

• Design and iterate multi-agent architectures that automate real accounting workflows end to end.

• Encode autonomy boundaries, tool usage, and fallback behaviors to keep agents safe and reliable.

• Manage context and memory for coherence across agent steps, with measurable success criteria.

• Route, evaluate, and optimize models under real-world constraints including latency, cost, and accuracy.

• Design and implement scalable evaluation pipelines (offline and online) that run hundreds of experiments automatically.

• Define golden tasks, labeling strategies, and metrics that make model performance measurable and comparable.

• Instrument the stack to detect regressions, track error taxonomies, and drive closed-loop improvement.

• Architect prompt stacks and retrieval pipelines; parse messy documents into structured representations for reasoning.

• Design guardrails and validation layers to keep agent behavior safe and deterministic.

• Communicate progress, learnings, and next steps clearly within your team pod.

What We're Looking For

• 3 or more years of applied AI/ML engineering experience at a senior level, beyond academic or junior-only backgrounds.

• Strong CS fundamentals; a background in math or physics is a plus.

• Degree in CS, Physics, Math, or another highly technical discipline.

• Deep proficiency in Python and hands-on experience with LLM and transformer-based systems.

• Experience building end-to-end LLM-based agent applications, including benchmarking, model orchestration, and evals for reliability.

• Experience designing and iterating multi-agent architectures.

• Experience architecting evaluation frameworks and structured feedback loops to measure and improve model performance.

• Experience building retrieval and indexing pipelines.

• Experience parsing messy, unstructured documents into structured representations.

• Previous hands-on experience at a fast-paced startup, a top-tier tech or AI company, or a hedge fund is strongly preferred.

• Excited to work in-office five days a week in New York.

Compensation & Benefits

Salary range: $175,000 to $300,000 USD annually. Visa sponsorship is available.

Location

On-site, five days per week in New York, NY, United States.