Data Engineer | Azure/Databricks
- Hiring from
- Portugal
- Work type
- Hybrid
- Posted
623,759 remote jobs, straight from company career pages
100% free · New jobs every minute
Show job descriptionHide job description
Make an impact by working in sectors where technology is the enabler and innovation is the key.
At Celfocus, we make data actionable through Next‑Gen Intelligence - where data, AI and human creativity come together.
You’ll work with multidisciplinary teams and partner with leading companies to turn data into decisions, agility and long‑term value.
Explore your way in.
The opportunity:
The Data Engineer is responsible for designing, building, and operating modern, scalable, and reliable data platforms that enable data-driven decision-making across the organization.
This role plays a critical part in transforming data into a strategic asset by ensuring that data is discoverable, trusted, secure, and accessible to business users, analysts, data scientists, and AI/ML applications.
The Data Engineer designs and implements robust data architectures, ingestion frameworks, processing pipelines, and storage solutions capable of supporting both traditional analytics and emerging Artificial Intelligence use cases, including Machine Learning and Generative AI.
The role requires a strong understanding of cloud-native technologies, distributed systems, data governance principles, and modern software engineering practices.
Maintaining a balance between technical excellence and business understanding is essential to ensure that data solutions generate measurable value for the organization.
Your day-to-day:
Data Platform Engineering
Design, develop, and maintain scalable, secure, and resilient data platforms in cloud environments.
Build and optimize batch and real-time data pipelines across multiple data sources and systems.
Implement and maintain modern data architectures such as Data Lakes, Lakehouses, Data Warehouses, and Data Mesh domains where applicable.
Develop reusable frameworks and standards for data ingestion, transformation, orchestration, and consumption.
Data Integration & Processing
Integrate structured, semi-structured, and unstructured data from databases, APIs, streaming platforms, SaaS applications, and external sources.
Develop and maintain ETL/ELT pipelines ensuring performance, scalability, and maintainability.
Design efficient data models supporting analytical, operational, and AI-driven workloads.
Implement data transformation and enrichment processes using distributed processing frameworks.
Data Quality, Governance & Security
Ensure high levels of data quality, reliability, lineage, and observability.
Define and implement data validation, monitoring, and alerting mechanisms.
Collaborate with governance teams to ensure compliance with security, privacy, and regulatory requirements.
Apply best practices in access management, encryption, data masking, and information protection.
AI & Advanced Analytics Enablement
Build and maintain data foundations that support Machine Learning, predictive analytics, and Generative AI solutions.
Prepare and optimize datasets for ML feature engineering and AI model consumption.
Support implementation of vector databases, embeddings pipelines, Retrieval Augmented Generation (RAG) architectures, and AI-ready data ecosystems when required.
Collaborate with Data Scientists, AI Engineers, and business stakeholders to operationalize AI use cases.
Platform Reliability & Operations
Monitor platform performance, reliability, and cost efficiency.
Troubleshoot and resolve data pipeline failures and platform issues.
Optimize data workloads for scalability, availability, and cloud resource consumption.
Promote DataOps practices through automation, testing, CI/CD, and Infrastructure as Code.
Collaboration & Innovation
Work closely with Product Owners, Business Analysts, Architects, Data Scientists, and other engineering teams.
Contribute to the evolution of data architecture standards and best practices.
Stay current with emerging technologies and assess opportunities to enhance the organization's data capabilities.
Actively contribute to building a data-driven culture across the organization.
What you'll bring:
Core Data Engineering
Strong proficiency in SQL and data modelling techniques (dimensional, normalized, Data Vault, etc.).
Strong programming skills in Python and/or Scala.
Experience designing and building large-scale data pipelines.
Experience with batch and streaming data processing architectures.
Cloud & Modern Data Platforms
Experience in at least one major cloud platform:
Microsoft Azure (mandatory)
AWS
Google Cloud Platform (GCP)
Examples of relevant technologies:
Azure: Data Factory, Data Lake, Synapse Analytics, Databricks, Fabric
AWS: S3, Glue, EMR, Lambda, Athena, Redshift, Lake Formation
GCP: BigQuery, Dataflow, Dataproc, Pub/Sub
Data Processing & Streaming
Experience with some of the following:
Apache Spark
Databricks (mandatory)
Apache Kafka
Apache Flink
Apache Beam
Airflow
dbt
Delta Lake
Apache Iceberg
Apache Hudi
Data Storage
Relational Databases (PostgreSQL, SQL Server, MySQL, Oracle)
NoSQL Databases (MongoDB, Cassandra, DynamoDB)
Data Warehouses and Lakehouse platforms
Object Storage solutions
DataOps & Software Engineering
Git and Git-based workflows
CI/CD pipelines
Infrastructure as Code (Terraform, CloudFormation, Bicep)
Containerization technologies (Docker)
Kubernetes fundamentals
Automated testing principles for data pipelines
Nice to have:
Azure, Databricks or Snowflake certifications.
Data Governance & Observability experience with: Data Catalogs, Metadata Management, Data Lineage;
Data Quality frameworks
Observability platforms
Master Data Management concepts Experience working in Data Mesh environments.
Experience supporting AI/ML platforms.
Experience with Data Governance programs.
Experience in FinOps and cloud cost optimization.
Experience mentoring junior engineers and contributing to engineering communities of practice.
What’s in it for you:
• Hybrid way of working with flexibility and balance
• Opportunity to work on international projects
• Possibility to explore different roles and career paths
• Continuous learning and development opportunities
• A growing AI‑driven environment, working with the latest technologies
• A set of benefits and perks to support your day-to-day
A place for everyone
At Celfocus, we are committed to cultivate a diverse and inclusive workplace. As an equal-opportunity employer, we welcome applicants of all backgrounds, gender identities, and abilities.
If you require any adjustments during the selection process, please inform our Talent Acquisition Team.
