GCP Senior Data Engineer

Purview Services; Chandigarh; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Purview Services seeks a data engineer to design, develop, and implement large-scale data pipelines using GCP services such as BigQuery, Pub/Sub, and Cloud Storage. You will collaborate with cross-functional teams to identify and prioritize project requirements and develop robust data models and architectures on GCP.

You will ensure high-quality data processing with testing and validation frameworks, optimize storage and retrieval for performance, and provide technical guidance on GCP best

Qualifications

• Solid data engineering fundamentals and hands-on GCP experience.
• Proficient with BigQuery, Pub/Sub and Cloud Storage.
• Experience with Hadoop, Spark and NoSQL databases.

Responsibilities

• Design, develop, and implement large-scale data pipelines on GCP.
• Collaborate with cross-functional teams to define requirements.
• Develop and maintain data models and architectures on GCP.
• Implement testing and validation to ensure data quality.
• Optimize storage and retrieval for performance and scalability.
• Provide guidance on GCP best practices to juniors.

Skills

GCP services

BigQuery

Pub/Sub

Cloud Storage

Data models

Testing frameworks

Performance optimization

Collaboration

Communication

Education

Bachelor's degree in Computer Science or related field

Tools

Hadoop

Spark

NoSQL

Job description

Roles and Responsibility

• Design, develop, and implement large-scale data pipelines using GCP services such as BigQuery, Pub/Sub, and Cloud Storage.
• Collaborate with cross-functional teams to identify and prioritize project requirements.
• Develop and maintain complex data models and architectures on GCP.
• Ensure high-quality data processing and analysis by implementing robust testing and validation frameworks.
• Optimize data storage and retrieval processes for performance and scalability.
• Provide technical guidance and support to junior team members on GCP best practices.

Job Requirements

• Strong understanding of data engineering principles and experience with GCP services.
• Experience with big data technologies such as Hadoop, Spark, and NoSQL databases.
• Proficiency in programming languages such as Python, Java, or Scala.
• Excellent problem-solving skills and attention to detail.
• Ability to work collaboratively in a fast-paced environment with multiple stakeholders.
• Strong communication and interpersonal skills to effectively interact with technical and non-technical audiences.