Senior Data Engineer

Senior Data Engineer (m/f/d)

Location: Germany | Remote

Duration: 12-Month Contract

Harvey Nash is looking for an experienced Senior Data Engineer to join a long-term data transformation project in Germany. The role focuses on designing and developing scalable cloud-based data platforms, data products, and modern ETL/ELT pipelines across Microsoft Azure and AWS.

You will work with complex data environments, supporting cloud migration and modernization initiatives while helping define robust data architectures, models, integration patterns, governance standards, and technical solutions.

Please note: This opportunity is intended for candidates based in Germany who have proficiency in German language, as the project involves collaboration with German-speaking stakeholders and teams.

Responsibilities

• Design and further develop enterprise-wide data architectures across AWS and Microsoft Azure.
• Define architecture principles, technical standards, and best practices for scalable data platforms and data products.
• Design solutions for Data Lakes, Data Warehouses, Data Hubs, and distributed data platforms.
• Develop integration architectures for heterogeneous source systems and large, complex datasets.
• Define and evaluate architecture decisions with a focus on scalability, performance, security, and maintainability.
• Create architecture concepts, technical specifications, data models, and data flows for development teams.
• Design and implement scalable ETL/ELT pipelines using Python, SQL, PySpark, and Apache Spark.
• Support the migration and modernization of legacy and on-premise data platforms into cloud-native environments.
• Define and improve standards for data quality, governance, security, and compliance.
• Analyse, integrate, model, and prepare master and transactional data from heterogeneous systems within rail freight transportation.
• Conduct architecture reviews and identify opportunities for modernization and cloud transformation.
• Design solutions for integrating AI and Machine Learning use cases into existing data platforms.
• Create and maintain architecture documentation and ensure technical decisions are properly documented and traceable.
• Collaborate with development teams, architects, business stakeholders, and other technical specialists throughout the project lifecycle.

Requirements

• 5+ years of practical experience developing data pipelines and data products in cloud environments.
• Strong hands-on experience with Python, SQL, PySpark, and Apache Spark.
• 5+ years of experience working with cloud-based data platforms across Microsoft Azure and AWS.
• Strong experience with Azure Data Factory, AWS Glue, and Amazon S3.
• Proven experience migrating and modernizing legacy/on-premise data platforms to cloud environments, including refactoring existing pipelines and workflows for cloud-native technologies.
• Strong experience in the analysis, integration, modelling, and preparation of master and transactional data.
• Experience working with data from rail freight transportation or closely related transportation/logistics environments.
• Ability to provide at least 3 relevant project references demonstrating experience with rail freight transportation data.
• Professional proficiency in German and English.
• Candidate must be based in Germany.

Nice to Have:

• Experience designing and implementing CI/CD pipelines, automated testing, development standards, and monitoring/logging solutions.
• Knowledge of data architecture, data product blueprints, Data Governance, Data Contracts, and cloud-based access/permission management.
• Experience working in Agile/SAFe environments.
• Hands-on experience with Jira and Confluence.
• Experience planning and conducting stakeholder workshops.
• Experience integrating Machine Learning and AI solutions into enterprise data platforms.

Why This Role?

This is an excellent opportunity for a Senior Data Engineer to contribute to a large-scale cloud transformation and data modernization project, working across AWS and Azure while helping shape enterprise data architecture and data products within the transportation and logistics domain.