Lead Data Engineer

Lead Data Engineer | £95,000 - £105,000 | Manchester or Warrington (Hybrid)

We're looking for a Lead Data Engineer to take ownership of the technical direction and continued development of a growing data platform.

A major part of the role will be leading the technical delivery of a large-scale data warehouse migration, moving an existing on-premise SQL Server environment towards a modern AWS-based lakehouse architecture.

You'll play a key role in shaping the platform's architecture, establishing engineering standards and driving improvements across data processing, streaming and overall platform performance.

This is a genuinely hands-on technical leadership position. You'll have the autonomy to make architectural decisions, challenge existing approaches and guide the wider engineering team, while remaining close to the code and day-to-day technical delivery.

What You'll Be Doing

• Leading the technical delivery of a major data warehouse migration from on-premise SQL Server to an AWS-based lakehouse.
• Defining the architecture, engineering standards and design patterns for the wider data platform.
• Designing and developing scalable batch and real-time data processing solutions.
• Building and improving data pipelines, ensuring they are reliable, efficient and maintainable.
• Developing real-time streaming capabilities to significantly reduce data processing and reporting latency.
• Establishing best practices around data modelling, data quality, validation and platform observability.
• Taking ownership of infrastructure-as-code and CI/CD standards across the platform.

What We're Looking For

• Significant experience in Data Engineering, with a proven track record of leading large-scale data platform or data warehouse migrations.
• Deep, hands-on experience with AWS data services, particularly Glue, Lambda, S3, Redshift and EMR.
• Advanced Python, PySpark and SQL skills.
• Proven experience building and operating real-time streaming architectures using Apache Kafka or Amazon MSK.
• Strong understanding of data warehousing, data modelling and modern lakehouse architecture.
• Experience with open table formats such as Apache Iceberg or Delta Lake.
• Experience with infrastructure-as-code, ideally Terraform, alongside CI/CD and pipeline orchestration.

If interested, please apply or reach out directly for a confidential chat!