Data Engineering Team Lead - Databricks

We are working with a leading global Financial Services business to hire an experienced Data Engineering Team Lead to head up a team of engineers working across a large-scale enterprise data platform. This is a senior hands-on leadership role requiring both deep technical expertise and the ability to drive delivery, mentor talent, and set standards across a distributed engineering function. Responsibilities: Lead, mentor, and develop a team of data engineers across multiple locations, driving code reviews, design reviews, and a culture of knowledge-sharing Own and drive Agile/Scrum delivery processes, ensuring the team operates effectively against roadmap priorities Design and develop scalable data solutions on a Lakehouse architecture platform, supporting enterprise-wide data processing and analytics Build, optimise, and maintain ETL/ELT pipelines and structured streaming workflows for both batch and real-time data ingestion Configure and tune clusters and Spark jobs to deliver consistent performance at scale Utilise Delta Live Tables and Unity Catalog to manage data ingestion, transformation, and access governance Apply IAM best practices and uphold compliance with data security and governance standards Support infrastructure provisioning and resource management using Terraform Implement monitoring frameworks covering pipeline performance, data quality, and operational health Contribute to technical documentation and promote continuous improvement across the engineering practice 8+ years in data engineering, with at least 3 years hands-on experience with Databricks Proven experience leading and managing a team of data engineers Strong Python and Spark programming skills Solid AWS experience across core services including S3, Glue, and Lambda Deep understanding of data modelling, SQL, and ETL/ELT design patterns Experience with Delta Lake, Lakehouse architecture, and Git-based version control Demonstrable use of AI tools within a professional development workflow Strong leadership, communication, and stakeholder management skills Desirable: Financial services or fund administration background Exposure to AI/ML implementation patterns and real-time data processing frameworks Multi-cloud experience beyond AWS API development or data governance framework experience Track record of developing junior and mid-level engineers in a fast-paced environment Salary: to be discussed, depending on experience Length: Permanent contract