Lead Data Platform Engineer

Overview We are seeking a Lead Data Platform Engineer to design, build, and optimize scalable data platforms and pipelines that support enterprise analytics and data-driven solutions. This role combines hands-on engineering with technical leadership, driving best practices in data architecture, platform performance, and governance while mentoring engineering teams. Key Responsibilities Design, develop, and maintain scalable data pipelines and ETL/ELT workflows. Build and optimize data platforms using Hadoop, Databricks, and cloud-based technologies. Integrate structured and semi-structured data into reliable, high-quality data solutions. Partner with cross-functional teams to translate business and analytics requirements into scalable engineering solutions. Lead technical design discussions and promote best practices in data modeling, performance optimization, and governance. Mentor data engineers and contribute to engineering standards, architecture, and platform scalability. Support innovation through proof-of-concepts, automation, and continuous platform improvements. 8+ years of experience in data engineering, including 2+ years in a technical leadership role. Strong Python skills (Pandas, NumPy, PySpark) and experience with Impala. Hands-on experience with Hadoop, Databricks, and large-scale data processing. Advanced SQL and experience with relational and distributed databases. Experience with cloud platforms such as Azure or AWS, including Databricks or Snowflake. Strong knowledge of ETL/ELT tools such as Apache Airflow, Apache NiFi, or Azure Data Factory. Experience with CI/CD, DevOps practices, and enterprise data platforms. Understanding of data modeling, governance, and performance optimization. Nice to Have Experience supporting AI/GenAI solutions through scalable data pipelines. Knowledge of machine learning workflows, feature engineering, and model serving. Experience processing unstructured data and implementing data governance, privacy, and security best practices. Strong analytical and problem-solving skills with the ability to communicate effectively across technical and business teams.