ai agent evaluation engineer remote at yo ai labs texas

Senior Software Engineer

**Job Type:**<br/>Contractor (:15 hours/week)

**Location:**<br/>Remote

**Job Summary**<br/>We are seeking experienced<br/>**Senior Software Engineers**<br/>to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using<br/>**Model Context Protocol (MCP)**<br/>tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as<br/>**bug fixing, feature implementation, codebase refactoring, and performance optimization**<br/>. No prior AI experience is required.

Key Responsibilities

- Create reinforcement learning environments for software engineering tasks.<br/>- Design tasks involving bug fixing, feature development, refactoring, and performance optimization.<br/>- Build deterministic verification systems and golden reference solutions.<br/>- Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.<br/>- Develop realistic, reproducible software engineering scenarios.<br/>- Ensure tasks accurately measure coding ability, problem-solving, and tool usage.<br/>- Document solutions and provide clear technical feedback.

**Required Skills**

- Strong proficiency in Python 3, Java, Rust, C++, or TypeScript.<br/>- Strong understanding of algorithms and data structures.<br/>- Experience with bug fixing and debugging complex software issues.<br/>- Proven experience in feature implementation and codebase refactoring.<br/>- Strong knowledge of performance optimization and tuning.<br/>- Excellent written and verbal communication.<br/>- Strong attention to detail.

**Preferred Qualifications**

- Experience working with large-scale or distributed codebases.<br/>- Familiarity with AI/ML systems is a plus but not required.<br/>- Experience with rigorous code reviews and software engineering best practices.<br/>- Experience working effectively in remote or cross-functional teams.

Hiring Process

- Submit an application and screening questions.<br/>- Complete an AI interview (:30 minutes).<br/>- Complete a technical assessment, if required.<br/>- Hiring Manager review.

**Compensation**<br/>Compensation is<br/>**output-based**<br/>, with payment provided per task that meets project specifications. Minimum weekly submission requirements may apply.

Availability

Selected experts should be prepared to begin their first tasks within<br/>**24–48 hours of completing onboarding**<br/>.

Location: Texas (Remote)

Remote: Yes