Senior Data Engineer

GS1 Egypt continuously develops its data capabilities to support its products, services and business decisions. We’re looking for a Senior Data Engineer to work with our data and engineering teams to deliver reliable data pipelines, improve data quality and support analytics and data science.

This is a senior individual contributor role requiring deep technical expertise and the ability to independently drive complex implementations through production operation. Working closely with the Data Architect, you will take ownership of engineering delivery, establish reusable practices and guide colleagues while remaining directly involved in development.

Responsibilities

• Design, build and maintain reliable data pipelines, warehouses and lakehouse solutions.

• Drive platform implementation in collaboration with the Data Architect, translating agreed designs into delivery plans, reusable components and production solutions.

• Integrate application databases, APIs, files and external sources using incremental ingestion, change tracking and reconciliation.

• Implement batch and streaming pipelines appropriate to business needs and agreed architecture.

• Develop and maintain shared data models that support business, product and regulatory requirements.

• Build reconciliation frameworks across heterogeneous sources, handling conflicting values, duplicates, late-arriving records and historical changes.

• Automate checks for completeness, validity, duplication, freshness and cross-source conflicts, with actionable exception reporting.

• Implement data cataloging, metadata management and lineage, connecting technical assets to business definitions and ownership.

• Embed data governance and regulatory requirements into pipelines and datasets, including schema validation, access controls and auditability.

• Prepare reliable datasets for business intelligence, regulatory reporting and data science, enabling self-service analytics through consistent metrics and governed access.

• Own pipeline monitoring, failure recovery, performance tuning and cloud consumption costs.

• Apply version control, automated testing, CI/CD and environment management.

• Evaluate engineering tools and implementation options against agreed architecture, workload requirements, maintainability and cost.

• Collaborate with analysts, data scientists and software engineers to deliver data solutions and automate repetitive work.

• Establish reusable engineering patterns, review technical work and mentor colleagues.

Requirements

• 7+ years of relevant experience in data engineering, data warehousing or data platforms, including hands-on technical leadership.

• Demonstrated ownership of complex data platform implementations, from agreed architecture through production delivery and ongoing operation.

• Strong SQL and Python skills, with practical experience in data modelling and ETL/ELT engineering.

• Strong hands-on experience with cloud data platforms, such as Microsoft Fabric, Azure data services, Databricks, Snowflake or equivalent AWS/GCP services, including integration, orchestration, storage, security, monitoring and cost optimization.

• Strong understanding of relational databases, incremental extraction, change data capture, schema evolution and historical data management.

• Strong experience with data reconciliation, including source-to-target validation, entity resolution, conflict handling and auditable exception management.

• Practical experience implementing data governance, including metadata management, cataloging, lineage, controlled access and auditable data handling.

• Experience enabling self-service BI through platforms such as Metabase, Apache Superset or Power BI, including data connectivity, analytical models, access controls and query performance.

• Experience with pipeline orchestration and dependency management using cloud-native services, Apache Airflow, Dagster or equivalent tools.

• Strong knowledge of dimensional modelling, shared canonical models and slowly changing dimensions.

• Experience with Git, automated testing and CI/CD for data workloads.

• Ability to independently investigate ambiguous problems, make defensible implementation decisions and deliver reliable solutions with limited supervision.

Nice to have

• Exposure to data mesh, data fabric, event-driven architecture, and Lambda or Kappa patterns.

• Experience with catalog and metadata tools such as OpenMetadata, DataHub, Apache Atlas or Microsoft Purview.

• Experience with regulated product data, sector-specific schemas or GS1 standards.

• Experience with distributed processing tools such as Apache Spark and open table formats such as Delta Lake, Apache Iceberg or Hudi.