Head of Evaluation & Model Quality (Expression of Interest)

Expression of Interest · Inquire First

We are meeting senior leaders for this planned function as we fundraise. This is an exploratory conversation, not an immediately funded salaried vacancy or a guaranteed start date. No cash salary is available today; any equity arrangement must be agreed separately. You are not being asked to begin work by expressing interest.

Use the Apply form below to introduce yourself and indicate your interest. We can discuss mutual fit, scope, funding, compensation, and timing before either side makes a commitment.

Remote · United States · Full time · Functional leadership

Determine when a market simulation deserves to influence a business decision. Build the evaluation function that makes model quality, failure modes, and release tradeoffs visible to research, product, and customers.

Why Subconscious

Subconscious is building the intelligence to run the economy. We start with a simulator for business decisions, using causal experiments on simulated markets to test products, pricing, messaging, and go-to-market choices before capital is committed. The long-term ambition is to help businesses discover unmet demand and build around what people actually need.

The function you will build

Own evaluation strategy, benchmark governance, regression criteria, and the model-quality hiring plan. Define the evidence required for release decisions and escalate unsupported claims. Partner with causal research on validity and with engineering on reliable execution of evaluations.

This is a hands-on leadership role. Set the functional roadmap and propose staffing and spending priorities with the founding team. Hiring and budget commitments depend on approved resources; an established team or funded budget is not assumed.

Proposed first 90 days after joining

Use these as an initial agenda, refined with the founders against the product, customer needs, and resources at your start.

• Inventory model claims and customer use cases; map each to a testable question and an appropriate reference or human baseline.

• Build reproducible evaluation sets with clear provenance, held-out testing, leakage controls, and uncertainty estimates.

• Establish failure analysis across populations, decision types, and distribution shifts, including cases where fluent output gives a misleading impression of quality.

• Create a release review that records tradeoffs among quality, latency, cost, and known limitations.

How we will judge progress

Evaluation results are reproducible, regressions are detected before release, and aggregate scores do not conceal important failures. Track coverage of consequential use cases, benchmark stability, error by relevant cohort, and time from diagnosis to a verified improvement.

What you bring

• You have led evaluation, measurement, or model-quality programs and can show decisions that changed because of the evidence.

• You are strong in statistics and practical ML evaluation, with hands-on ability to build data and analysis pipelines.

• You can challenge a promising result constructively, investigate confounding or leakage, and explain why a metric is or is not useful.

We are looking for evidence of judgment, execution, and leadership. A particular company pedigree or previous executive title is not a substitute for that evidence.

Compensation and funding

We are actively fundraising. Only equity-based compensation is currently available; no cash salary is available today. The equity grant and its terms have not been set for this role and must be agreed before joining. Any future salary, incentive plan, benefits, hiring budget, or paid start date depends on financing and written agreement. Funding and its timing are not guaranteed.

Before deciding to join, discuss the grant structure, fully diluted ownership basis, vesting, dilution, applicable exercise terms, company funding position, and working arrangement. A company valuation is not the cash value of an equity grant.

How we work

Remote · United States. The planned role is full time; timing is to be agreed. Expect direct collaboration with the founders, work across scientific and commercial boundaries, and responsibility for doing the early work yourself. The current role does not include visa sponsorship. Confirm practical collaboration hours and any travel expectations during the initial conversation.

Start with work you already own

Describe an evaluation that contradicted an attractive headline result. How did you establish the failure, and what changed in the model, product, or release decision?

Please do not share confidential employer or customer materials. Bring your questions about our product, evidence, team, finances, and the authority attached to the role. Mutual diligence matters for a decision of this size.