Senior Data Engineer

Job Title: Azure Data Engineer

Location: Brussels, Belgium (Hybrid)

Duration: 12 Months

Experience: Minimum 5 Years

Job Description

We are looking for an experienced Azure Data Engineer with strong expertise in Azure Databricks and Unity Catalog to design, develop, and maintain scalable data solutions on Microsoft Azure.

The ideal candidate will have hands-on experience in building data pipelines, implementing modern data architectures, managing data governance, and optimizing platform performance and costs.

Key Responsibilities

• Design, develop, and maintain data pipelines using Azure Data Factory, Azure Databricks, and other Azure services.
• Implement modern data architectures, including Data Lake, Delta Lake, and Lakehouse.
• Develop and optimize ETL/ELT processes for data ingestion and transformation.
• Implement and manage Databricks Unity Catalog, including catalogs, schemas, permissions, and data governance.
• Support self-service data capabilities while ensuring data security and governance.
• Monitor and optimize Azure and Databricks performance, resource consumption, and costs.
• Automate data ingestion, transformation, and deployment processes.
• Collaborate with Data Analysts, Data Scientists, and business stakeholders.
• Document data flows, architecture, and technical best practices.
• Ensure data platform scalability, reliability, and performance.

Required Technical Skills

• Minimum 5 years of experience in Data Engineering.
• Strong experience with Microsoft Azure Data Architecture.
• Hands-on experience with Azure Data Factory (ADF).
• Strong expertise in Azure Databricks and Unity Catalog.
• Good knowledge of Apache Spark, PySpark, and Delta Lake.
• Strong programming skills in Python and SQL.
• Experience with Azure Storage, Azure SQL, Key Vault, and Azure DevOps.
• Experience with data governance, access management, and security.
• Knowledge of CI/CD principles and data pipeline automation.
• Experience with performance monitoring and cost optimization.
• Knowledge of REST APIs and AMQP integration patterns.

Nice to Have

• Microsoft Azure or Databricks certifications.
• Experience with Power BI administration and governance.
• Experience with real-time data streaming.
• Knowledge of Terraform or Infrastructure as Code (IaC).
• Familiarity with Azure Monitor and Log Analytics.
• Basic knowledge of SAP S/4HANA or BW/4HANA.
• Experience working in Agile/Scrum environments.

Candidate Profile

• Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
• Strong analytical and problem-solving skills.
• Ability to work independently and collaboratively.
• Good communication skills and ability to explain technical concepts clearly.

Interested candidates with strong Azure Databricks and Unity Catalog experience are encouraged to apply.