We are working with an independent technology consultancy that works across data, cloud and technology transformation, helping clients modernise their platforms and make better use of their data.
About The Role
As part of a major data modernisation programme, you will be responsible for designing, building and optimising the data flows that power a new cloud-based platform.
The role will focus heavily on data ingestion, transformation and modelling, with Snowflake as the core data warehouse and Kafka supporting real-time data capture and processing.
This is an opportunity for a Data Engineer who enjoys working with real-time data and solving technical challenges around streaming, integration and large-scale data transformation.
Responsibilities
Design, build and maintain scalable ELT/ETL data pipelines within Snowflake.
Implement data ingestion from a range of sources, including real-time streams using Kafka.
Develop and deploy KSQL and KStreams pipelines for real-time data processing.
Develop efficient and scalable data models within Snowflake to support analytics and business intelligence.
Ensure data quality, integrity and reliability throughout the data lifecycle.
Optimise data pipelines and Snowflake workloads for performance and scalability.
Work closely with architects, analysts and data scientists to understand requirements and deliver effective data solutions.
Contribute to data engineering standards, best practices and continuous improvement across the platform.
Essential Skills & Experience
Solid commercial experience as a Data Engineer, with a track record of delivering cloud-based data solutions.
Previous experience within insurance or financial services.
Strong knowledge of Snowflake, including data modelling and performance optimisation.
Hands-on experience with Kafka for data ingestion and stream processing.
Practical experience with KStreams and KSQL pipeline development and deployment.
Strong SQL skills and experience with Python or another scripting language.
Experience with data orchestration tools such as Airflow.
Experience working with at least one major cloud platform, such as AWS, Azure or GCP.
Strong understanding of data ingestion, transformation, modelling and integration.
Ability to work collaboratively with technical and non-technical stakeholders.
Desirable Skills & Experience
Experience building data pipelines for Policy Administration Systems, particularly Guidewire.
Understanding of data schemas and Confluent Schema Registry.
Experience working with CI/CD practices and DevOps teams.
Broader experience across AWS, Azure or GCP.
Benefits
Salary up to €60,000.
Health insurance and mental health support.
Financial support for ongoing professional training and development.
Flexible working, with hybrid working available for Lisbon-based employees and remote working available elsewhere in Portugal.
Opportunities to work on modern data and cloud transformation programmes.
Career development opportunities within a growing technology consultancy.