Manager - IT Quality & Assurance Testing

Company:

Description:

We are seeking a talented individual to join our Tech Team at Marsh. This role will be based in Gurugram.This is a hybrid role that has a requirement of working at least three days a week in the office.This role will be based in Gurgaon. This is a hybrid role that has a requirement of working at least three days a week in the office

Manager – Quality / AI Assurance

We will count on you to:

A visionary forward thinker who views Quality holistically and can drive a Quality Engineering mindset across traditional software and AI-powered applications. Understanding industry trends, tools and practices and implementing the right approaches to deliver high-quality, scalable and reliable software.

• 10+ years' experience in Software Testing / Quality Engineering with experience in managing end-to-end project delivery.
• Proven experience leading QA teams across Manual, Automation and AI/LLM testing.
• Strong understanding of STLC, SDLC, Agile, test strategy, test planning, risk management, defect management and quality metrics.
• Experience defining testing strategies for Web/UI, API, Database, AI/LLM, RAG and AI-powered applications.
• Strong automation experience using Python, JavaScript/TypeScript, Java, Playwright, Selenium and WebdriverIO (WDIO).
• Good understanding of API testing, SQL, relational databases and data validation.
• Understanding of RAG architecture, embeddings, vector databases, semantic search, retrieval and reranking.
• Experience with Generative AI, LLM evaluation, prompt engineering, AI agents and AI safety testing.
• Experience with AI/LLM testing and evaluation frameworks such as PromptFoo, Ragas, DeepEval, LangSmith and PyRIT, or similar tools.
• Strong understanding of AI quality and security risks including prompt injection, jailbreaks, hallucination, data leakage, guardrails and responsible AI.
• Experience with modern DevOps and test automation practices, including Git, GitHub, GitHub Actions and CI/CD pipeline integration.
• Strong communication, stakeholder management, analytical and people leadership skills.

What you need to have:

• Input into planning and subsequent resource planning – including skills profiling and identifying training requirements for team members.
• Define and drive the Quality Engineering, Automation and AI Testing strategy across projects.
• Lead testing across UI, API, Database, Automation and AI/LLM applications.
• Establish AI testing and evaluation practices covering accuracy, relevance, faithfulness, groundedness, hallucination, safety and security.
• Drive testing of RAG applications, vector databases, retrieval pipelines, prompts and AI agent workflows.
• Evaluate and adopt appropriate AI testing, evaluation, observability and security tools based on project requirements.
• Lead automation initiatives using Playwright, WebdriverIO, JavaScript/TypeScript and Java, integrated with CI/CD and GitHub Actions.
• Establish scalable and maintainable automation frameworks, reusable utilities and test assets.
• Define and monitor test coverage, automation coverage, AI evaluation metrics, defect leakage and release quality.
• Drive shift-left testing, continuous testing, risk-based testing and quality engineering practices.
• Ensure effective root cause analysis, defect prevention and continuous quality improvement.
• Mentor and develop QA engineers across manual testing, automation and AI testing.
• Work closely with Product, Engineering, AI/ML, Data Science, Security and Business teams.
• Manage team performance, delivery risks, conflicts and stakeholder expectations.
• Ensure consistent testing standards, processes and quality practices across teams.

What makes you stand out:

• Minimum 12 months experience as Senior Principal Engineer
• Should have at least “On-Track” or equivalent rating in last appraisal.
• Should not be under any Performance Counselling Plan or have any official disciplinary action within the last 3 month

• Strong understanding of Quality Engineering, Manual Testing, Automation and AI/LLM Testing.
• Strong experience with Playwright, WebdriverIO/Selenium, Python, JavaScript/TypeScript and Java.
• Good knowledge of UI, API, SQL/Database and data validation testing.
• Understanding of RAG, embeddings, vector databases, semantic search, retrieval and LLM evaluation.
• Experience with AI/LLM evaluation and testing tools such as PromptFoo, Ragas, DeepEval, LangSmith and PyRIT.
• Understanding of AI security, prompt injection, jailbreaks, hallucination, guardrails, red-team testing and responsible AI.
• Experience with GitHub and CI/CD pipelines.
• Experience building and maintaining scalable automation frameworks.
• Proven experience managing, mentoring and developing QA/Quality Engineering teams.
• Strong communication, stakeholder management, analytical and problem-solving skills.
• Ability to evaluate emerging technologies and introduce effective testing solutions.
• Passion for AI, automation, Quality Engineering and continuous improvement.

Why join our team:

• We help you be your best through professional development opportunities, interesting work and supportive leaders.
• We foster a vibrant and inclusive culture where you can work with talented colleagues to create new solutions and have impact for colleagues, clients and communities.
• Our scale enables us to provide a range of career opportunities, as well as benefits and rewards to enhance your well-being.