Relomote
Brillio logo

Lead Quality Engineer

BrillioEngineering & Development
Hiring from
India
Work type
Hybrid
Posted

625,840 remote jobs, straight from company career pages

100% free · New jobs every minute

Show job description
Consultant

Primary Skills

  • CI/CD Pipeline, React (React Context / Redux), Django / Flask / FastAPI, Docker, CSS/SCSS/LESS, PyTest, PostgreSQL / MongoDB / Redis, Kubernetes, REST API / GraphQL

    • Job requirements

  • Role: QA Engineer – GenAI Agents and GitLab Duo • Design and execute functional, integration, regression, security, and performance tests for GenAI agents and multi-agent workflows. • Validate application and infrastructure code interrogation results for accuracy, evidence, severity, false positives, and false negatives. • Test GitLab Duo, AI Catalog agents, shared CI/CD templates, pipeline triggers, reports, and security gates. • Develop automated Python-based test suites and evaluation datasets to assess agent reliability, consistency, security, and overall quality. Key Responsibilities 1. AI Agent and Workflow Testing • Create and execute test plans for GenAI agents and agentic workflows. • Validate single-agent, multi-agent, and orchestrated workflows. • Verify agent instructions, tool usage, task routing, handoffs, and termination conditions. • Test agent behaviour for valid, invalid, incomplete, and ambiguous inputs. • Validate error handling, retry mechanisms, timeouts, and fallback scenarios. 2. Application Code Interrogation Testing Validate the application code interrogation agent against: • Security vulnerabilities and OWASP Top 10 risks. • Insecure coding practices. • Improper authentication and authorization. • Sensitive-data exposure and secret leakage. • Missing input validation and error handling. • Resiliency issues such as missing timeouts, retries, bulkheads, and circuit breakers. • Incorrect logging, monitoring, and exception-handling practices. Verify that each finding contains accurate: • File name and line number. • Issue description and technical evidence. • Severity and confidence. • Business impact. • Recommended remediation. • Classification as confirmed or potential finding. 3. Infrastructure Code Interrogation Testing Test the infrastructure interrogation agent against Terraform, Kubernetes, Helm, YAML, and CI/CD configuration files. Validate detection of: • Single points of failure. • Inadequate high availability and scaling. • Insecure storage and missing encryption. • Excessive network access. • Unrestricted communication between security zones. • Missing identity and access controls. • Inadequate logging, monitoring, and alerting. • Insecure secrets management. • Misconfigured cloud and container resources. 4. SDLC Agent Validation Support testing of AI agents used across the software development lifecycle, including: • Requirement and story decomposition. • UX/UI specification generation. • OpenAPI contract generation and governance. • Architecture validation. • Backend and frontend implementation assistance. • Dependency and open-source package curation. • Unit, integration, contract, and end-to-end test generation. • Security review. • Infrastructure planning and implementation. 5. Triage and Finding Validation Validate the interrogation-triage agent for: • Finding normalization. • Duplicate detection and deduplication. • Correct categorization. • Severity and priority assignment. • Ranking based on exploitability, exposure, and blast radius. • Correlation of findings from application and infrastructure agents. • Accurate consolidation of related findings. • Avoidance of false positives and unsupported conclusions. 6. GitLab Dedicated CI/CD Integration Testing • Validate agent execution through GitLab Dedicated pipelines. • Test integration with shared CI/CD templates. • Validate pipeline triggers for pushes, merge requests, and scheduled executions. • Verify pipeline status, security gates, and failure behavior. • Validate generated reports, artifacts, logs, and auditability. • Test repository permissions and agent access controls. • Verify behavior across different programming languages, repository structures, and infrastructure projects. 7. Test Automation • Develop automated test cases using Python and suitable testing frameworks. • Build reusable test datasets containing secure, vulnerable, and intentionally incorrect code. • Automate regression testing for prompts, tools, agents, and

    • Similar jobs

    Apply for this job100% free