Senior Data Engineer (Databricks & Cloud Data Platforms)

DVT is one of Africa’s leading software development consultancies, partnering with top organisations across South Africa and globally to deliver innovative, enterprise-grade technology solutions.

At DVT, our engineers work on complex, high-impact projects using modern cloud and data technologies while collaborating with some of the strongest technical talent in the industry. We are passionate about continuous learning, technical excellence, and growing our people through mentorship, technical communities, sponsored events, and ongoing professional development.

We are looking for a Senior Data Engineer with deep Databricks expertise to help design and deliver modern, scalable data platforms for enterprise clients.

Job Description: Senior Data Engineer (Databricks & Cloud Data Platforms)

Position Overview

As a Senior Data Engineer, you will play a critical consulting role in designing, building, and modernising enterprise data platforms, with a strong focus on Databricks lakehouse implementations across cloud environments.

You will work closely with solution architects, analysts, data scientists, and client stakeholders to deliver secure, scalable, and high-performing analytics platforms that drive real business impact.This is a hands-on technical role suited to someone passionate about solving complex data challenges and delivering best-in-class cloud-native solutions.

What You’ll Be Doing

· Design and implement scalable Databricks-based data platforms

· Build robust ETL / ELT pipelines for batch and streaming workloads

· Lead data migration and modernisation initiatives from legacy platforms to cloud-based lakehouse architectures

· Develop high-performance data processing solutions using PySpark, Python, and SQL

· Implement Delta Lake, Unity Catalog, and modern governance frameworks including Purview

· Design enterprise-grade lakehouse and medallion architectures

· Optimise Spark workloads for performance, scalability, and cost efficiency

· Build orchestration solutions using Databricks Workflows, ADF, Airflow, or similar tools in Azure/AWS

· Collaborate with technical and business stakeholders to define solution requirements

· Mentor junior engineers and contribute to technical best practices within the Data & AI practice

· Produce technical documentation, architecture artefacts, and reusable delivery assets

Technical Requirements

· Databricks Expertise (Essential)

· Cloud Data Platform Experience (Azure Fabric, AWS) (Essential)

You are:

· A strong technical leader who can own complex delivery outcomes

· Comfortable engaging directly with clients and stakeholders

· A proactive problem-solver with excellent analytical skills

· Able to clearly communicate technical concepts to both technical and non-technical audiences

· Passionate about mentoring and uplifting engineering teams

· Delivery-focused, quality-driven, and adaptable in consulting environments

Minimum Experience

· 8+ years in Data Engineering projects

· 3+ years of hands-on Databricks delivery experience

· Deep expertise in Python and PySpark

· Strong understanding of the Medallion Architecture and Lakehouse Architecture

· Hands-on experience building and maintaining live Databricks AI/BI Governance Dashboards using SQL

· Strong experience with Azure data services including Synapse, ADF, OneLake, Purview

· Practical exposure to AWS data platforms including Glue, S3, Redshift

· Proven track record delivering data migration and modernisation projects

· Experience working in consulting or client-facing environments

· Bachelor’s degree in computer science, Engineering, Information Systems, or equivalent practical experience

Why Join DVT?

At DVT, you’ll have the opportunity to:

· Work on enterprise-scale data transformation projects

· Collaborate with top-tier engineering talent

· Expand your expertise across modern cloud and data platforms

· Grow through continuous learning, mentorship, and technical leadership opportunities

· Be part of a people-first engineering culture focused on innovation and excellence