Data Engineer

At SWAG Live's data team, our goal is to democratize data by building a self-service data platform. We strive to empower our internal teams with the ability to access, understand, and generate valuable insights from data.

In this role, you'll help develop data-driven decision-making tools for various stakeholders. These stakeholders may include software engineers, data analysts, and data scientists, among others. Your work will focus on building a unified data platform, aligning with SWAG Live's data mesh vision.

Responsibilities

1. Design, develop, and support pipelines and reporting warehouses to solve business, operations, and product problems.
2. Query optimization, schema design, and ELT/ETL pipelining.
3. Stream processing in event-driven architectures.
4. Collaborate with product and business stakeholders to ensure our data infrastructure meets the constantly evolving requirements.
5. Build developer tools to better support data scientists and data analysts.

Requirements

1. Proficient in Python and SQL.
2. In-depth knowledge for transactional DB and data warehouses.
3. Hands-on experience in query optimization and schema design.
4. Fundamental understanding of analytical data modeling (dbt).
5. Sufficient experience with one of the popular orchestration frameworks (airflow , kubeflow , dagster , prefect, argo).
6. Experience in stream processing (apache spark, apache beam, kafka , flink).
7. A self-directed learning mentality.
8. Comfortable working in an English-speaking environment.

Good to have

1. Basic knowledge (or interest) in common machine learning algorithms.
2. Have hands-on DevOps experience and value DevEx, especially focusing on GitOps.
3. Proficiency in the software development lifecycle and system design, particularly pertaining to data architecture.