Senior Data Engineer

We are hiring a Senior Data Engineer who owns the data platform end to end, not a queue of SQL tickets.
You join the client's data team and work directly with their Chief Solutions Architect. The team has built, from scratch, a templated and repeatable CDC framework that moves operational data from MySQL and PostgreSQL into Snowflake.
This is not a role for someone who waits for detailed technical specs. The client gives clear business context and expects the engineers to define the technical path, break work into tasks themselves and communicate trade-offs openly.

Requirements:

• 5+ years in data engineering, with production experience building and running data pipelines for a real product;
• Hands-on experience designing CDC solutions, ideally with Debezium on MySQL and/or PostgreSQL;
• Strong Kafka experience (Confluent Platform or Confluent Cloud): Kafka Connect, SMT framework, topics and partitioning, schema registry, delivery semantics;
• Experience with Apache Flink or another stream processing framework;
• Solid Snowflake experience, including Snowpipe, tasks and streams;
• Apache Airflow: building DAGs, scheduling, dependencies, failure handling;
• Strong SQL and RDBMS design: normalisation, indexing, query performance, schema evolution;
• Python for pipelines and automation;
• General familiarity with Microsoft Azure;
• Engineering fundamentals for data: idempotency, ordering, exactly-once vs at-least-once, late and duplicate events, backfills and reprocessing;
• Ownership and agency: you set priorities when guidance is thin, notice gaps nobody owns and close them;
• You have built data pipelines from scratch and made them templated and repeatable, not one-off scripts;
• You plan, estimate and deliver what you commit to, and give honest feedback on technical risks and project challenges;
• English at an advanced level (B2/C1 or equivalent): you can discuss design with a US technical lead, disagree respectfully and write documentation people actually read.

Nice to have

• Databricks experience or certification;
• dbt or similar transformation tooling;
• Infrastructure as code (Terraform, Bicep) and CI/CD for data pipelines;
• Background in logistics, delivery or supply chain data.

Responsibilities:

• Design and evolve CDC pipelines that stream changes from MySQL and PostgreSQL through Debezium and Confluent/Kafka into Snowflake;
• Extend the existing templated pipeline framework to onboard new data sources quickly and consistently;
• Handle schema changes in source databases without breaking downstream consumers;
• Build stream processing and transformations with Kafka Connect SMTs and Apache Flink;
• Design and implement automated ETL/ELT pipelines and orchestrate them with Apache Airflow;
• Automate ingestion and processing inside Snowflake with Snowpipe, tasks and streams;
• Design and run data archiving: move historical data out of large MySQL tables into Snowflake safely, then automate the clean-up;
• Define reprocessing and backfill strategies, and own reliability: monitoring, alerting, data quality checks and incident follow-ups;
• Tune performance and keep cloud and Snowflake costs in check;
• Build data models for reporting and new business cases;
• Work in Microsoft Azure: networking, VMs, storage and access for the data platform;
• Turn high-level business requirements into technical tasks, estimate them, deliver on commitments, and present strategies and results to the client;
• Document decisions and pipelines so the platform does not depend on one person's memory;
• Use AI tools as part of your workflow and verify what they produce.

What we offer

• Real ownership: you work directly with the client's technical lead, not through layers of coordination;
• A long-term engagement with an established US company in a growing domain;
• An international team, minimal bureaucracy, and short paths to decisions;
• An engineering culture that expects and supports serious use of AI tools;
• Flexible working arrangements, remote by default;
• Room to grow into technical leadership as the engagement expands.

About the project
Our partner specialises in logistics delivery management, managing the final mile delivery of products. The solution integrates with multiple ERPs, ordering and internal systems, and its algorithms turn operational data into decisions for the people who run deliveries. The platform is growing with new large enterprise carrier contracts, and the data layer is what keeps every system in sync.