Big Data Developer

Thompsons HR Consulting LLP; Chennai; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

A leading IT consulting firm is seeking a Big Data Developer in Chennai, with 7+ years of experience in big data technologies. Responsibilities include designing scalable data warehouses, maintaining data pipelines, and collaborating with business stakeholders. Proficiency in Google Cloud Platform and programming in Python or Java is necessary. This full-time role offers an opportunity to advance in a vibrant team environment.

Qualifications

• 7+ years of experience in big data technologies.
• Strong knowledge of data processing frameworks.
• Proficiency in programming languages like Python, Java, or Scala.

Responsibilities

• Design and develop scalable data warehouses using BigQuery.
• Build and maintain batch and streaming data pipelines using Apache Beam.
• Automate and orchestrate jobs for scheduled data workflows.

Skills

Google Cloud Platform (GCP)

Distributed systems

Data processing frameworks

Big data architecture

Programming languages: Python, Java, Scala

Tools

BigQuery

Dataflow

Dataproc

Cloud Spanner

Job description

3 days ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

• Strong knowledge of distributed systems, data processing frameworks, and big data architecture.
• Proficiency in programming languages like Python, Java, or Scala.
  

Role: Big Data Developer

Location: Chennai

Experience: 7+ years

Work Mode : Work from Office

Key Skills Required

• Google Cloud Platform (GCP)
• BigQuery (BQ)
• Dataflow
• Dataproc
• Cloud Spanner
• Strong knowledge of distributed systems, data processing frameworks, and big data architecture.
• Proficiency in programming languages like Python, Java, or Scala.
  

Roles And Responsibilities

• BigQuery (BQ):
• Design and develop scalable data warehouses using BigQuery.
• Optimize SQL queries for performance and cost-efficiency in BigQuery.
• Implement data partitioning and clustering strategies.
• Dataflow:
• Build and maintain batch and streaming data pipelines using Apache Beam on GCP Dataflow.
• Ensure data transformation, enrichment, and cleansing as per business needs.
• Monitor and troubleshoot pipeline performance issues.
• Dataproc:
• Develop and manage Spark and Hadoop jobs on GCP Dataproc.
• Perform ETL/ELT operations using PySpark, Hive, or other tools.
• Automate and orchestrate jobs for scheduled data workflows.
• Cloud Spanner:
• Design and manage globally distributed, scalable transactional databases using Cloud Spanner.
• Optimize schema and query design for performance and reliability.
• Implement high availability and disaster recovery strategies.
• General Responsibilities:
• Collaborate with data architects, analysts, and business stakeholders to understand data requirements.
• Implement data quality and data governance best practices.
• Ensure security and compliance with GCP data handling standards.
• Participate in code reviews, CI/CD deployments, and Agile development cycles.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Engineering and Information Technology

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at Thompsons HR Consulting LLP by 2x

Sign in to set job alerts for “Big Data Developer” roles.

Software Development Internship in Chennai

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.