Middle Data Quality Engineer/BA
We are seeking a detail-oriented and technically skilled Middle Data Quality Engineer / Business Analyst to join a project for a niche American insurance company with 40+ years of experience in credit insurance, undergoing digital transformation.
The team is focused on migrating and modernizing a legacy data platform to Azure Synapse Analytics. This project entails migrating complex data and business logic from legacy SQL/Access-based systems into a robust Bronze/Silver/Gold medallion data architecture. You will collaborate closely with Data Engineers and technical leads to rebuild data processing pipelines, enhance business logic, automate data workflows, and ensure high standards of data quality and reconciliation.
In this role you will deal with:
- Preparing comprehensive test scenarios and detailed test cases for migrated data and transformations
- Validating Bronze-layer data against source systems to ensure accuracy and completeness
- Performing source-to-target data reconciliation across Bronze, Silver, and Gold layers
- Validating record counts, field values, transformations, duplicates, and overall data completeness
- Investigating data discrepancies between source and target systems
- Defining and implementing validation rules and acceptance criteria for data migration
- Maintain thorough test evidence and clearly document all test results and findings
- Preparing and updating technical and testing documentation throughout the project lifecycle
- Collaborating with Data Engineers and the Lead Data Engineer to clarify requirements and resolve technical issues
- Participating in defect analysis, retesting, and regression testing activities as needed.
We'd love to hear from you if you have:
- Hands-on experience with data quality assurance, data QA, or ETL/data testing (3+ years)
- Solid knowledge of SQL, with practical ability to write and optimize queries for data validation and reconciliation
- Applied experience in data migration testing or ETL/ELT testing
- Solid experience validating data between source and target systems
- Exposure to data quality dimensions such as completeness, accuracy, consistency, uniqueness, and validity
- On-the-job experience in creating and executing test cases and test scenarios
- Solid experience in data reconciliation, including comparison of record counts and field-level data
- Ability to investigate and diagnose data discrepancies, distinguishing issues arising from source data, transformation logic, or target systems
- Strong attention to detail and proven ability to work effectively with large and complex datasets
- Practical application in preparing and maintaining clear technical and testing documentation
- Ability to work independently and communicate findings and issues effectively to developers and technical leads
- Good English communication skills for documentation and stakeholder interaction.
Nice to have:
- Exposure to Azure Synapse Analytics and Azure Data Factory
- Basic understanding of testing PySpark, data lake, and lakehouse solutions
- Familiarity with SQL Server and Azure SQL
- Awareness of Medallion data architectures.