Human IT logo

Senior Data Engineer (Apache Flink/Kafka/AWS) - Hybrid Lisbon (3 days/week office)

Hiring from
Portugal
Work type
Hybrid
Posted

625,803 remote jobs, straight from company career pages

100% free · New jobs every minute

Show job description

Senior Data Engineer (Apache Flink/Kafka/AWS) - Hybrid Lisbon (3 days/week office)

ABOUT THE OPPORTUNITY

We're looking for a Senior Data Engineer to join a Group Data Delivery team that builds real-time data products on a group-wide data platform. You'll own your data products end to end, from ingestion and stream processing to APIs, monitoring and production support. You'll also have room to set engineering standards and mentor other engineers. If you enjoy building streaming systems with Apache Flink and Apache Kafka on AWS, this role puts you at the core of the platform.

PROJECT & CONTEXT

The team builds real-time and batch data products on an event-driven architecture, and keeps improving them after they reach production. Your work will include:

  • Designing and building real-time and batch data pipelines with Apache Flink and Apache Kafka
  • Developing event-driven services, microservices, REST APIs and OpenAPI specifications
  • Delivering data products that are reliable, secure, observable and production-ready
  • Implementing data quality controls, schema management, validation and automated testing
  • Setting up CI/CD pipelines for build, test, deployment and rollback
  • Implementing monitoring, logging, metrics, tracing and alerting with CloudWatch, Prometheus, Grafana and OpenTelemetry
  • Supporting production services, investigating incidents, improving performance and contributing to root-cause analysis
  • Applying cloud security practices: authentication, authorisation, encryption, secrets management and IAM
  • Producing technical designs, API documentation, deployment documentation and operational runbooks
  • Reviewing technical work, defining engineering standards and mentoring other engineers

WHAT WE'RE LOOKING FOR (Required)

  • 7+ years of experience in data engineering
  • Proven experience delivering real-time data products in production
  • Strong hands-on experience with Apache Flink and Apache Kafka
  • Experience with AWS data and application services
  • Good knowledge of APIs, microservices, event-driven systems and schema management
  • Experience with CI/CD, automated testing, observability and production support
  • Ability to turn business requirements into clear technical designs
  • Experience reviewing implementation quality and supporting less experienced engineers
  • English at B2 level or above (professional working proficiency)
  • Based in Lisbon and available to work on-site 3 days per week

NICE TO HAVE (Preferred)

  • Hands-on experience with AWS streaming services such as Amazon MSK, Kinesis or Managed Service for Apache Flink
  • Experience with container platforms on AWS (ECS/EKS)
  • Experience with schema registries and schema evolution strategies
  • Experience implementing OpenTelemetry-based distributed tracing

Similar jobs

Apply for this job100% free