Lead Data Engineer

Purview Services; Jaipur; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Purview Services in Jaipur, India is seeking a Data Engineer to design and implement large-scale data pipelines using GCP services like BigQuery, Pub/Sub and Cloud Storage.

You will collaborate with cross-functional teams to translate requirements into scalable data models and architectures, ensure robust validation, and guide junior engineers on GCP best practices. Strong Python/Java/Scala skills are essential in a fast-paced environment.

Qualifications

• Strong understanding of data engineering principles.
• Experience with big data technologies.
• Proficiency in Python/Java/Scala.
• Excellent problem-solving and collaboration.

Responsibilities

• Design, develop, and implement large-scale data pipelines using GCP services such as BigQuery, Pub/Sub, and Cloud Storage.
• Collaborate with cross-functional teams to identify and prioritize project requirements.
• Develop and maintain complex data models and architectures on GCP.
• Ensure high-quality data processing and analysis by implementing robust testing and validation frameworks.
• Optimize data storage and retrieval processes for performance and scalability.
• Provide technical guidance and support to junior team members on GCP best practices.

Skills

Python

Java

Scala

Tools

BigQuery

Pub/Sub

Cloud Storage

Hadoop

Spark

NoSQL

Job description

Roles and Responsibility

• Design, develop, and implement large-scale data pipelines using GCP services such as BigQuery, Pub/Sub, and Cloud Storage.
• Collaborate with cross-functional teams to identify and prioritize project requirements.
• Develop and maintain complex data models and architectures on GCP.
• Ensure high-quality data processing and analysis by implementing robust testing and validation frameworks.
• Optimize data storage and retrieval processes for performance and scalability.
• Provide technical guidance and support to junior team members on GCP best practices.

Job Requirements

• Strong understanding of data engineering principles and experience with GCP services.
• Experience with big data technologies such as Hadoop, Spark, and NoSQL databases.
• Proficiency in programming languages such as Python, Java, or Scala.
• Excellent problem-solving skills and attention to detail.
• Ability to work collaboratively in a fast-paced environment with multiple stakeholders.
• Strong communication and interpersonal skills to effectively interact with technical and non-technical audiences.