Cloud Data Engineer

We are looking for a Cloud Data Engineer, who will be responsible for planning, design, developing and maintaining data models, data pipelines and standards for various Data Integration & Data Lake and Data Warehouse projects in the AWS Cloud. Ensure new features and subject areas are modelled to integrate with existing structures and provide a consistent view. Develop and maintain documentation of the data architecture, data flow, and data models of the data warehouse appropriate for various audiences. Adopt Cloud technologies and industry best practices in the field of data lakes and data warehouse architecture and modelling.
The role is based in our Gurugram, India location which has a flexible hybrid work model that has employees in-office 2-3 days per week and 2-3 days per week remote.
This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project.

• Demonstrated ability to have successfully complete multiple, complex technical data engineering projects and create high-level design and architecture of the solution.
• Able to communicate the capabilities and improvements offered by data solutions with both the development teams and business user teams.
• Participate in technical design discussions and implement solutions along with the team and assist in building/maintaining medallion data architecture.
• Develop and maintain data pipelines using AWS cloud-native data platforms and services following established patterns and team standards.
• Design, build and maintain robust ETL/ELT pipelines from multiple sources following the best practices.
• Design data pipes and automate data ingestion and cleansing, transformation activities using the DevOps methodologies.
• Contribute to migration of legacy pipelines to modern data architecture.
• Write complex SQL/python/Pyspark transformations and build data models that support analytics, reporting and downstream applications.
• Monitor, optimize and troubleshoot data pipelines to ensure reliability, scalability, security and compliance with the data quality and governance standards.
• Collaborate with cross-functional project team to deliver end-to-end solutions.

• Minimum 3 years of experience working as a Data Engineer
• Must have experience with cloud-based data platforms, data lakes, cloud data warehouse (Redshift or Snowflake).
• Proven direct experience with traditional databases, RedShift, AWS Glue, AWS Bedrock, Athena, AWS Lambda, Sagemaker, Lake Formation, IAM, SQS, S3 and other AWS core services.
• Expertise in SQL and write optimized SQLs across platforms.
• Expertise in writing python scripts and ability to manipulate data using Pandas/Pyspark. Familiarity and usage of different file formats in batch processing like Parquet/ORC and Open table formats.
• Experience integrating data from CRM, ERP, and marketing platforms (Salesforce, Eloqua, Oracle) into data lake/warehouse.
• Experience in data modelling, ELT, implementing complex stored Procedures and standard DWH and ETL/ELT concepts.
• Experience in advanced concepts like setting up resource monitors, Role-based access controls (RBAC), query performance tuning, time travel and understanding how to use these features.
• Experience in deploying features such as data sharing, events, and lake-house patterns.
• Provide resolution to an extensive range of complicated data pipeline related problems, proactively and as issues surface.
• Experience with data security and data access controls and design.
• Strong analytical, troubleshooting, and critical thinking skills.
• Familiarity with Devops tools for version control, CI/CD, GitHub.
• Exposure to Enterprise BI tools like Power BI and ThoughtSpot.
• Experience developing and supporting production data pipelines.
• Exposure to AI and leveraging AI tools for daily tasks.
• Effective communication skills with ability to translate complex technical work to business users.