Big Data Engineer

Role summary

We’re looking for a Big Data Engineer to build and maintain scalable data pipelines and support analytics and ML work.

• Design, build, and maintain ETL/ELT pipelines for large-scale data
• Optimize Spark jobs for performance and reliability
• Prepare data for analytics and ML use cases
• Work with stakeholders to understand data needs

Requirements

• Strong SQL and data modeling skills
• Apache Spark (DataFrame API, Spark SQL, performance tuning)
• Python for data engineering and basic data science tasks
• Building and maintaining ETL/ELT pipelines
• Working with large-scale structured and semi-structured data
• Data lake concepts (Parquet, Delta Lake, partitioning)
• Git and CI/CD fundamentals
• Cloud data platforms (Azure, AWS; most of our work is on Azure)
• Basic statistics, data analysis, and data quality practices
• Troubleshooting and performance optimization
• Good communication and stakeholder collaboration skills

Nice to have

• Databricks experience
• Exposure to ML projects