GCP Data Engineer

Job Summary

We are seeking an experienced GCP Data Engineer with 8+ years of experience in designing, building, and optimizing scalable data pipelines on the Google Cloud Platform (GCP). The ideal candidate should have strong expertise in PySpark, Cloud Composer, BigQuery, and Data Governance, with hands-on experience in building enterprise-grade data engineering solutions for large-scale data processing and analytics.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using GCP services.
• Build and optimize ETL/ELT workflows using PySpark.
• Develop and manage data orchestration workflows using Cloud Composer (Apache Airflow).
• Design and optimize data warehouses using BigQuery.
• Implement data governance practices, including data quality, metadata management, security, and compliance.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
• Monitor and optimize data pipeline performance for scalability and reliability.
• Ensure adherence to coding standards, security policies, and best practices.
• Troubleshoot and resolve production issues related to data pipelines and cloud infrastructure.
• Participate in code reviews, documentation, and knowledge-sharing sessions.
  
  

Mandatory Skills

• 8+ years of experience in Data Engineering.
• Strong expertise in Google Cloud Platform (GCP).
• Hands-on experience with PySpark.
• Experience with Cloud Composer (Apache Airflow).
• Strong knowledge of BigQuery.
• Experience implementing Data Governance frameworks and best practices.
• Strong SQL and data modeling skills.
• Experience designing and optimizing ETL/ELT pipelines.
• Knowledge of cloud-native data architectures.
• Experience with Git and version control systems.
  
  

Preferred Skills

• Experience with other GCP services such as Cloud Storage, Pub/Sub, Dataflow, Dataproc, and IAM.
• Knowledge of CI/CD pipelines for data engineering workflows.
• Experience with data lakes and modern data warehouse architectures.
• Exposure to DevOps practices and Infrastructure as Code (IaC).
  
  

Educational Qualification

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
  
  

Required Competencies

• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management.
• Ability to work independently in a remote/offshore environment.
• Strong collaboration and documentation skills.
• Ability to deliver high-quality solutions within project timelines.
  
  

Primary Skills

GCP | PySpark | Cloud Composer | BigQuery | Data Governance

Skills: cloud composer,data governance,pyspark,bigquery,gcp