Data Engineer ID94415

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

We are looking for a Data Engineer to execute large-scale data platform migrations to Snowflake and support reliable Airflow workflows.

- Strong software engineering fundamentals.

- 3+ years of hands-on Python development experience.

- Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.

- Snowflake data modeling.

- Apache Airflow — developing and testing pipelines in Airflow and Python.

- Apache Kafka.

- Apache Spark, including hands-on PySpark experience.

- Iceberg and Parquet data formats.

- AWS experience, particularly S3 (data lakes, partitioning, and lifecycle policies).

- Testing strategy experience, with the ability to hands-on debug and test code.

- At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.

- Overlap until 12 PM Pacific Time.

- Upper-intermediate English level.

- Hands-on experience with Snowflake migrations (data warehouse modernization, performance and cost optimization).

- Experience with Apache Airflow migrations (version upgrades, executor changes, or platform migrations).

- Experience using AI tools for development.

- Work embedded within the data engineering function, partnering closely with the data platform team to support a large-scale data platform modernization effort.

- Execute the migration of approximately 1,000 datasets to Snowflake.

- Adapt existing pipelines to Snowflake, following the core architecture and technical approach defined by the internal team.

- Ensure data quality and reliability throughout the migration process.

- Support workflow scheduling and monitoring in Apache Airflow 3.

- Execute time-sensitive migrations, including Airflow 2 to 3, Confluent region migration, and the move to Iceberg.

- Help accelerate the migration initiatives targeted for completion by the end of March 2027.

- Professional growth: Mentorship, TechTalks, and personalized growth roadmaps.

- Competitive compensation: USD-based pay with education, fitness, and team activity budgets.

- Exciting projects: Modern solutions with Fortune 500 and top product companies.

- Flextime: Flexible schedule with remote and office options.

Application → Coding Challenge → Video Interview → Technical Interview or Hiring Manager Interview

Each step helps us understand your skills and overall fit.

If it’s a match, you’ll receive an offer.