Sr. Data Engineer (Python, PySpark, Azure Databricks)
- Hiring from
- Canada
- Work type
- Remote
- Posted
623,547 remote jobs, straight from company career pages
100% free · New jobs every minute
Show job descriptionHide job description
Position Name – Sr. Data Engineer
Type of hiring – Fulltime
Location – Remote Canada
Looking for a Sr Data Engineer who has experience in AI & Claude, Python, PySpark, Azure Databricks, Spark SQL, Dataframes, Datasets, Real-time Data Streaming, Azure Data Services & Advanced SQL.
Job Description:
Role Overview
- 5-8 Years of hands-on experience in software development, with a strong emphasis on Python programming.
- Experience in AI (Artificial Intelligence) & Claude code is mandatory.
- Deep expertise in PySpark for building and optimizing scalable data pipelines.
- Proficient in designing and implementing real-time data streaming solutions using Spark SQL, Dataframes, and Datasets.
- Solid experience with Azure Databricks, including working with Delta tables, workflows, and scripting in Python.
- Familiar with Azure cloud services, especially Azure Data Lake, Azure Data Factory, and EventHub or Kafka for streaming data ingestion.
- Skilled in writing complex SQL queries and building views involving joins across multiple tables.
- Experience across the Software Development Lifecycle (SDLC), with a strong grasp of Agile Scrum methodologies.
- Proficient in tools like Jira (project management), Confluence (documentation), and GitHub (version control and CI/CD pipelines).
Soft Skills & Collaboration:
- Strong collaboration mindset with enthusiasm for mobbing sessions and cross-team coordination.
- Excellent problem-solving and code review skills.
- Strong communication skills with the ability to understand business requirements and translate them into technical solutions.
- Proactive and responsive during business hours for any ad-hoc calls or meetings.
- Demonstrated ability to prioritize effectively, think creatively, and adapt quickly to new technologies and feedback.
