Data Engineering Lead

At Vanguard X, we are looking for a Data Engineering Lead to join an international AdTech company and take ownership of its analytics and reporting platform.

This is a hands-on, player-coach role combining technical leadership, people management, and production engineering. You will lead a team of Data Engineers while remaining actively involved in architecture, coding, data modeling, and solving complex data challenges.

The platform processes high-volume, near-real-time event data, with Kafka and ClickHouse at its core.

100% Remote

The team operates across the United States and Australia, so regular and agreed working-hour overlap with both regions will be required.

What you'll do

• Lead and mentor Data Engineers, setting technical direction, priorities, and delivery standards.
• Design, build, and operate high-volume ETL/ELT and near-real-time data pipelines.
• Work hands-on with Kafka, including producers, consumers, event design, replay, and troubleshooting.
• Design and optimize ClickHouse schemas, queries, partitioning, sharding, replication, and performance.
• Maintain the current data pipeline while leading the phased modernization of the legacy architecture.
• Build reliable aggregation, reprocessing, and backfill workflows.
• Define standards for data quality, testing, monitoring, observability, and documentation.
• Partner with Product, Engineering, Analytics, and leadership to translate business and reporting needs into scalable data solutions.
• Investigate data discrepancies affecting revenue, attribution, optimization, and customer reporting.

What we're looking for

• 6+ years of experience building and operating production data systems.
• Experience with high-volume event data and near-real-time pipelines.
• Previous experience leading or directly managing Data Engineers while remaining hands-on.
• Strong production experience with ClickHouse or a comparable columnar OLAP database; direct ClickHouse experience is strongly preferred.
• Strong hands-on experience with Apache Kafka.
• Advanced SQL and strong data modeling skills.
• Experience with distributed data systems, including partitioning, sharding, replication, and performance optimization.
• Experience with CI/CD, orchestration, observability, and data quality practices.
• Working knowledge of AdTech data flows and metrics such as impressions, clicks, conversions, attribution, CPM, RPM, and CTR.
• Strong communication and problem-solving skills.