Senior Data Engineer with Manufacturing

Data Engineer

Limerick (Remote)

Contract - B2B

JD

Key Responsibilities

• Analyse and profile raw manufacturing and operational data from the external site, identifying source-system structures, data-quality issues, dependencies, and onboarding requirements.

• Design, build, test, and maintain scalable Python-based Databricks ETL notebooks and pipelines for ingestion, transformation, validation, and publishing.

• Create reusable CDL bronze and silver asset-centric datasets that conform to enterprise data architecture, engineering, metadata, security, and quality standards.

• Develop a flexible master data framework that consistently aligns products, materials, equipment, processes, batches, and relevant manufacturing reference data.

• Build governed KPI calculation frameworks for yield, cycle time, process deviations, and other agreed product and manufacturing-process performance measures.

• Develop aggregated performance models that support trend analysis and reporting over time, enabling insights into operational efficiency and product performance.

• Partner with manufacturing, quality, business, analytics, and source-system subject-matter experts to define requirements, validate business rules, and confirm the fitness of delivered datasets and metrics.

• Implement automated data-quality checks, reconciliation, monitoring, logging, and exception handling to ensure reliable and traceable data products.

• Optimise pipeline performance, reusability, and maintainability, and document source mappings, transformation logic, data lineage, KPI definitions, and operational support procedures.

• Work within agile delivery and DevOps practices, including source control, peer review, automated testing, release management, and production support.

• Provide technical leadership, engineering guidance, and mentoring while promoting consistent standards and reusable patterns across the delivery team.

Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, Information Technology, Engineering, or a related discipline; equivalent relevant experience will also be considered.

• Typically 7 or more years of relevant data engineering experience, including ownership of complex data onboarding and modelling deliveries.

• Advanced hands-on Python and SQL skills, with strong software-engineering practices for modular, testable, supportable data solutions.

• Strong practical experience with Databricks, Apache Spark, notebook-based development, Delta Lake or equivalent lakehouse patterns, and performance optimisation.

• Demonstrated experience designing scalable ETL/ELT pipelines and layered bronze and silver data products using reusable engineering patterns.

• Experience designing master data, dimensional, asset-centric, aggregation, and time-series models for analytics and reporting.

• Strong understanding of data quality, metadata, lineage, reconciliation, observability, access controls, and production support.

• Experience working with manufacturing and operational data, including batch, product, material, equipment, process, quality, and performance data.

• Experience with manufacturing execution systems (MES), SAP, historians, laboratory or quality systems, and industrial analytics environments is highly desirable.

• Knowledge of manufacturing KPIs and the underlying business rules for yield, cycle time, deviations, process performance, and trend-based analysis is highly desirable.

• Experience with source control, CI/CD, automated testing, agile delivery, and regulated or validated data environments is preferred.

• Strong analytical, problem-solving, stakeholder-management, and written and verbal communication skills, with the ability to translate operational requirements into robust technical solutions.

• Proven ability to work across internal teams and external-site stakeholders, influence technical decisions, and deliver effectively with limited supervision.

• Core technical capabilities: Python, SQL, Databricks, Apache Spark, lakehouse and Delta patterns, ETL/ELT pipeline engineering, data modelling, master data, data quality, metadata and lineage, Git-based source control, CI/CD, and cloud data-platform delivery.