Generative AI Engineer

Job title- AI Engineer (Generative AI & Agent Quality)

Open for C2C/W2

H1B Transfer Support Available

LOCATION is remote

Application Engineer Senior (7+ Years)

AI Engineer (Generative AI & Agent Quality)

Position Summary

We are seeking an AI Engineer to help design, test, optimize, and support AI-powered agent experiences across customer and internal business workflows. This role combines prompt engineering, agent orchestration, quality assurance, evaluation framework development, and AI observability.

The ideal candidate is curious, analytical, and hands-on, with experience working with Large Language Models (LLMs), prompt design, testing methodologies, and modern software development practices. You will partner closely with product managers, software engineers, architects, and business stakeholders to improve agent behavior, reliability, accuracy, and user experience.

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
• 7-10 years of experience in software engineering, QA engineering, automation testing, business systems analysis, or related technical roles.
• 1+ years of hands-on experience with Generative AI, LLMs, prompt engineering, or conversational AI solutions.
• Experience working with Azure DevOps or similar Agile delivery platforms.
• Experience designing and executing structured test plans.
• Strong analytical and troubleshooting skills.
• Experience working with APIs and JSON data structures.
• Excellent written and verbal communication skills.

Preferred Qualifications

• Experience with Arize AI or comparable AI observability tools.
• Experience with Azure OpenAI, OpenAI, Anthropic, Gemini, or other LLM platforms.
• Knowledge of Retrieval-Augmented Generation (RAG) architectures.
• Experience evaluating AI systems using human and automated scoring methods.
• Familiarity with vector databases and semantic search.
• Experience creating automated test frameworks.
• Understanding of responsible AI principles and AI governance.
• Experience with Python, SQL, JavaScript, or C#.
• Knowledge of prompt versioning and experimentation practices.

Technical Skills

AI & GenAI

• Prompt Engineering
• Agent Design
• AI Evaluation Frameworks
• RAG Concepts
• LLM Testing
• AI Quality Assurance
• AI Observability
• Prompt Optimization

Tools

• Arize AI
• Azure DevOps
• Azure OpenAI
• Git
• Postman
• Jira (optional)
• Power BI (optional)

Development

• Python
• SQL
• REST APIs
• JSON
• Test Automation Frameworks