3 hours ago
Remote Senior Big Data Engineer (Python)
Full time
- Senior
- Remote
- Warszawa, PL
PythonJavaData ScienceC#C++ScalaSparkKafkaAWSGCPAzure
37,307 – 40,291 PLN
Important: if an employer asks you to log into their system via iCloud or Google, send a code, an SMS or Telegram password, run some code, or install software — refuse. These are signs of fraud.
Job Description
We are looking for Data Engineers to work remotely for an Adtech company that leverages machine learning and data science to build an identity graph that can scale to reach millions of users via brands with programmatically selected households. The work includes scaling our Big Data asset that combines billions of transaction data points including intent, conversions, first party data into an identity graph.We value technical excellence and you will have both resources and time to deliver world-class code.This is a 100% remote position, You will be working with team members in NYC.If you like solving hard and technically challenging problems, join us to use those skills here to create real-time, concurrent, globally distributed systems applications and services.Role details:Design, develop, and maintain highly scalable data pipelines, ETL processes, and data models using Python, Spark, and other big data technologies in a cloud environment (AWS/GCP).Lead the integration of AI and LLMs into our data products, working with data scientists to productionize machine learning models and agentic workflows in an efficient and ethical fashion.Champion and utilize AI-driven software development tools (e.g., GitHub Copilot) to boost development productivity, improve code quality, and accelerate
- on creating and maintaining reliable and scalable distributed data processing systemsBecome a core maintainer of the data lakeMaintain our data lake by building searchable data sets for broader business usesScale, troubleshoot and fix existing applications and servicesOwn a complex set of services and applicationsFocus ensuring that our data pipelines run 24/7Lead technical discussions leading to improvements in tools, processes or projectsWork on scaling our identity graph to deliver impactful advertising campaignsWork on data sets exceeding billions of recordsScale our MLOps platform by using both traditional ML as well as LLM/Generative AI based applicationsKey requirements:
- 8+ years of professional software engineering experience, with a focus on data engineering in big data environments.
- Expert-level proficiency in Python and at least one other high-level programming language (e.g., Java, Scala, C++, C#).
- Proven experience building and optimizing large-scale data pipelines using Apache Spark.
- Hands-on experience developing and deploying data solutions in a major cloud platform (AWS, GCP, or Azure).
- Experience working with AI, LLMs, Agents, and/or Generative AI technologies, both in product applications and for development productivity.
- Relevant technology experience working in the advertising industry.
- Experience with Scala
- Experience with big data visualization and analytics using OLAP tools.
- Familiarity with big data tools and frameworks such as MLFlow, dbt, Kafka, Airflow or Databricks.
- Experience with large-scale data management formats like Parquet, Delta Lake, or Iceberg.Recruitment process:Short call with
- Initial call with hiring manager or team member Panel interview with tech team (includes live coding/live debugging session)Panel interview with product team (culture/team fit check)Final decision (CTO meeting)
Match
37%
Good match
We match every vacancy against your profile and show a fit score — so you instantly know which ones are worth applying to. Sign up and create a resume — it's free.
We are looking for Data Engineers to work remotely for an Adtech company that leverages machine learning and data science to build an identity graph that can scale to reach millions of users via brands with programmatically selected households. The work includes scaling our Big Data asset that combines billions of transaction data points including intent, conversions, first party data into an identity graph.We value technical excellence and you will have both resources and time to deliver world-class code.This is a 100% remote position, You will be working with team members in NYC.If you like solving hard and technically challenging problems, join us to use those skills here to create real-time, concurrent, globally distributed systems applications and services.Role details:Design, develop, and maintain highly scalable data pipelines, ETL processes, and data models using Python, Spark, and other big data technologies in a cloud environment (AWS/GCP).Lead the integration of AI and LLMs into our data products, working with data scientists to productionize machine learning models and agentic workflows in an efficient and ethical fashion.Champion and utilize AI-driven software development tools (e.g., GitHub Copilot) to boost development productivity, improve code quality, and accelerate
- on creating and maintaining reliable and scalable distributed data processing systemsBecome a core maintainer of the data lakeMaintain our data lake by building searchable data sets for broader business usesScale, troubleshoot and fix existing applications and servicesOwn a complex set of services and applicationsFocus ensuring that our data pipelines run 24/7Lead technical discussions leading to improvements in tools, processes or projectsWork on scaling our identity graph to deliver impactful advertising campaignsWork on data sets exceeding billions of recordsScale our MLOps platform by using both traditional ML as well as LLM/Generative AI based applicationsKey requirements:
- 8+ years of professional software engineering experience, with a focus on data engineering in big data environments.
- Expert-level proficiency in Python and at least one other high-level programming language (e.g., Java, Scala, C++, C#).
- Proven experience building and optimizing large-scale data pipelines using Apache Spark.
- Hands-on experience developing and deploying data solutions in a major cloud platform (AWS, GCP, or Azure).
- Experience working with AI, LLMs, Agents, and/or Generative AI technologies, both in product applications and for development productivity.
- Relevant technology experience working in the advertising industry.
- Experience with Scala
- Experience with big data visualization and analytics using OLAP tools.
- Familiarity with big data tools and frameworks such as MLFlow, dbt, Kafka, Airflow or Databricks.
- Experience with large-scale data management formats like Parquet, Delta Lake, or Iceberg.Recruitment process:Short call with
- Initial call with hiring manager or team member Panel interview with tech team (includes live coding/live debugging session)Panel interview with product team (culture/team fit check)Final decision (CTO meeting)