Opening For - Big Data Hadoop

SP Staffing; India; Hybrid; INR 1,200,000 - 2,400,000; Full time

Job summary

SP Staffing is seeking an experienced Data Engineer to design, develop, and optimize large-scale data platforms. The role emphasizes building scalable pipelines, batch and streaming processing, and close collaboration with stakeholders.

The ideal candidate brings expertise in Big Data technologies, distributed computing, and modern data engineering practices to deliver reliable data solutions for business needs across Pan India. Virtual interview process may be involved.

Qualifications

• Strong expertise in Big Data technologies and modern data engineering practices.
• Experience building scalable data platforms and collaborating with stakeholders.
• Familiarity with batch and streaming data processing.

Responsibilities

• Design, develop and maintain enterprise-scale data ingestion and processing pipelines.
• Develop batch and streaming solutions using Hadoop, Spark, Kafka and NiFi.
• Build scalable ETL/ELT frameworks to process structured, semi-structured and unstructured data.
• Design and implement real-time data streaming solutions using Kafka and Spark Streaming.
• Develop data integration workflows using Apache NiFi.
• Optimize data processing jobs for performance, scalability and reliability.
• Collaborate with Data Architects and Data Modelers to implement data models and platform architecture.
• Implement data quality, governance and monitoring frameworks.
• Troubleshoot production issues and perform root cause analysis.
• Ensure adherence to security, compliance and data governance standards.
• Participate in code reviews and technical design discussions.

Skills

Big Data

Real-time processing

Distributed computing

Data integration

Stakeholder collaboration

Tools

Hadoop

Spark

Kafka

NiFi

Spark Streaming

Job description

Greetings from SP Staffing Services!

Reaching out to you regarding permanent MNC opportunity!!

Job Description:

Location: Pan India

Experience : 5 Yrs- 15 Yrs

Interview Mode:Virtual Interview

Interview Process: Technical & Managerial / HR round

We are seeking an experienced Data Engineer to join our team and contribute to the design development and optimization of largescale data platforms The ideal candidate should have strong expertise in Big Data technologies realtime data processing distributed computing and modern data engineering practices The role requires working closely with business stakeholders solution architects data scientists and engineering teams to build scalable and highperformance data solutions

Key Responsibilities

• Design develop and maintain enterprisescale data ingestion and processing pipelines
• Develop batch and streaming solutions using Hadoop Spark Kafka and NiFi
• Build scalable ETLELT frameworks to process structured semistructured and unstructured data
• Design and implement realtime data streaming solutions using Kafka and Spark Streaming
• Develop data integration workflows using Apache NiFi
• Optimize data processing jobs for performance scalability and reliability
• Collaborate with Data Architects and Data Modelers to implement data models and platform architecture
• Implement data quality reconciliation monitoring and governance frameworks
• Troubleshoot production issues and perform root cause analysis
• Ensure adherence to security compliance and data governance standards
• Participate in code reviews architectural discussions and technical design activities