Big Data Hadoop

Intellics Global Services; Bengaluru; On-site; INR 1,200,000 - 2,100,000; Full time

Job summary

Intellics Global Services in Bengaluru, India is seeking an experienced Data Engineer to design, develop and optimize enterprise-scale data pipelines. You will work with batch and streaming tech to process structured and unstructured data, implement data quality and governance, and collaborate with architects and data scientists to deliver scalable solutions.

Applicants should be proficient with Hadoop, Spark, Kafka and NiFi, and comfortable solving performance, reliability and security

Qualifications

• Experience building and maintaining enterprise-scale data pipelines.
• Strong knowledge of batch and streaming data processing.
• Experience with data governance and data quality practices.

Responsibilities

• Design, develop and maintain enterprise-scale data ingestion and processing pipelines.
• Develop batch and streaming solutions using Hadoop Spark Kafka and NiFi.
• Build scalable ETLELT frameworks to process structured semi structured and unstructured data.
• Design and implement real time data streaming solutions using Kafka and Spark Streaming.
• Develop data integration workflows using Apache NiFi.
• Optimize data processing jobs for performance scalability and reliability.
• Collaborate with Data Architects and Data Modelers to implement data models and platform architecture.
• Implement data quality reconciliation monitoring and governance frameworks.
• Troubleshoot production issues and perform root cause analysis.
• Ensure adherence to security compliance and data governance standards.
• Participate in code reviews architectural discussions and technical design activities.

Skills

Big Data

Real-time processing

Distributed computing

Data governance

Tools

Hadoop

Spark

Kafka

NiFi

Job description

Role & responsibilities

• Design develop and maintain enterprise scale data ingestion and processing pipelines
• Develop batch and streaming solutions using Hadoop Spark Kafka and NiFi
• Build scalable ETLELT frameworks to process structured semi structured and unstructured data
• Design and implement real time data streaming solutions using Kafka and Spark Streaming
• Develop data integration workflows using Apache NiFi
• Optimize data processing jobs for performance scalability and reliability
• Collaborate with Data Architects and Data Modelers to implement data models and platform architecture
• Implement data quality reconciliation monitoring and governance frameworks
• Troubleshoot production issues and perform root cause analysis
• Ensure adherence to security compliance and data governance standards
• Participate in code reviews architectural discussions and technical design activities

Preferred candidate profile

We are seeking an experienced Data Engineer to join our team in Denmark and contribute to the design development and optimization of largescale data platforms The ideal candidate should have strong expertise in Big Data technologies real time data processing distributed computing and modern data engineering practices The role requires working closely with business stakeholders solution architects data scientists and engineering teams to build scalable and high performance data solutions