Big Data Developer

Viraaj HR Solutions Private Limited; Maharashtra; On-site; INR 1,000,000 - 1,500,000; Full time

Benefits offered by this job

Collaborative engineering culture

Competitive compensation

Training in cloud and Big Data technologies

Job summary

A staffing and HR services firm is hiring a Big Data Engineer in India to design, build, and optimize data pipelines for mission-critical applications. The ideal candidate should have strong skills in Apache Spark, Hadoop, SQL, and Python, with a Bachelor's degree in a related field. The role includes developing ETL jobs and streamlining data workflows in a collaborative engineering culture.

Qualifications

• Proven experience delivering on-site in enterprise Big Data environments.
• Familiarity with production deployment practices and CI/CD for data workflows.
• Strong troubleshooting mindset with experience in performance tuning and data quality assurance.

Responsibilities

• Design and implement end-to-end batch and streaming data pipelines using Spark and Hadoop.
• Develop ETL jobs and data transformation logic in Python/SQL.
• Integrate real-time data streams using Kafka and ensure low-latency delivery.

Skills

Apache Spark

Hadoop HDFS

Apache Hive

Apache Kafka

SQL

Python

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Apache Airflow

AWS EMR

Scala

Job description

A staffing and HR services firm focused on placing talent into Enterprise IT, Analytics, and Big Data engineering projects across Banking, Retail, Telecom, and SaaS sectors. We partner with product and services organisations to deliver scalable data platforms, real-time analytics, and high-throughput ETL solutions.

We are hiring an on-site Big Data Engineer in India to design, build, and optimise data pipelines and platform components for mission-critical data applications.

Role & Responsibilities

• Design and implement end-to-end batch and streaming data pipelines using Spark and Hadoop ecosystem components.
• Develop ETL jobs and data transformation logic in Python/SQL, ensuring scalability and maintainability.
• Integrate and process real-time data streams using Kafka and related ingestion tools; ensure low-latency delivery to downstream systems.
• Author Hive queries and optimize performance for large-scale datasets; tune Spark jobs to reduce cost and improve throughput.
• Collaborate with data engineers, analysts, and stakeholders to translate business requirements into robust data models and pipelines.
• Implement monitoring, logging, and automated testing for data workflows; troubleshoot production issues and perform root-cause analysis.

Skills & Qualifications

Must-Have

• Apache Spark
• Hadoop HDFS
• Apache Hive
• Apache Kafka
• SQL
• Python

Preferred

• Scala
• Apache Airflow
• AWS EMR

Other Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field (or equivalent practical experience).
• Proven experience delivering on-site in enterprise Big Data environments; familiarity with production deployment practices and CI/CD for data workflows.
• Strong troubleshooting mindset with experience in performance tuning and data quality assurance.

Benefits & Culture Highlights

• On-site delivery model with high-impact projects for large enterprise clients and opportunities to work end-to-end on data platforms.
• Collaborative engineering culture, technical mentoring, and exposure to modern data stack components.
• Competitive compensation and opportunities for certification/training in cloud and Big Data technologies.