Cloudera Hadoop Engineer (Big Data Engineer)

Dixp; Delhi; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Dixp is seeking a Cloudera Hadoop Engineer in Delhi to design, develop, and optimize large-scale data pipelines on distributed clusters. The role requires strong experience with Hadoop ecosystem technologies including Hive, Impala, Spark, Airflow, and HDFS, and hands-on work with CDP in enterprise environments.

The ideal candidate will collaborate with data architects, administrators, and analytics teams to deliver efficient data ingestion, processing, and governance across cloud/on-premise

Qualifications

• 4–7 years of experience in Big Data / Hadoop development.
• Experience with Cloudera Data Platform (CDP) and related Hadoop tools.
• Strong knowledge of Hive, Impala, Spark, and data ingestion workflows.

Responsibilities

• Design and develop data pipelines and ETL workflows using Hadoop ecosystem tools.
• Build and optimize large-scale data processing jobs on distributed clusters.
• Process structured and unstructured datasets using Hive, Spark, and Impala.

Skills

Hadoop ecosystem

Hive

Impala

Spark

CDP

ETL pipelines

SQL

Airflow

HDFS

Education

B.Tech/B.E. in Computer Science/Engineering/Data Engineering

Job description

Experience: 4 - 7 yrs

Job Type: Full-Time / Contract

Education:

• UG: B.Tech/B.E. in Computer Science, Engineering, Data Engineering, or a related field

Job Description

Project Role Description: We are looking for a Cloudera Hadoop Engineer with strong experience in the Hadoop ecosystem and Cloudera Data Platform (CDP) to build and manage large-scale data processing pipelines. The candidate will work closely with data architects, administrators, and analytics teams to design, develop, and optimize big data workflows and data ingestion pipelines on distributed clusters. The role requires expertise in Hadoop ecosystem technologies such as Hive, Impala, Spark, Airflow, and HDFS, along with experience working on Cloudera-based big data environments.

Key Responsibilities:

• Design and develop data pipelines and ETL workflows using Hadoop ecosystem tools.
• Build and optimize large-scale data processing jobs on distributed clusters.
• Process structured and unstructured datasets using Hive, Spark, and Impala.

Cloudera Platform Development

• Develop and manage workloads on Cloudera Data Platform (CDP).
• Work with services such as Hive, Impala, HDFS, Airflow, and Hue.
• Optimize queries and workloads running on the Cloudera ecosystem.

Data Ingestion & Processing

• Design pipelines for data ingestion from multiple sources including databases, APIs, and files.
• Implement batch and scheduled data workflows using Airflow or similar orchestration tools.
• Ensure data quality, transformation, and efficient storage in distributed systems.

Performance Optimization

• Tune Hive and Impala queries for performance and scalability.
• Optimize data partitioning, indexing, and storage formats for big data processing.
• Monitor and improve performance of data pipelines.
• Work with data scientists, analysts, and business teams to understand data requirements.
• Integrate big data solutions with analytics platforms and reporting systems.
• Collaborate with Cloudera administrators to ensure cluster efficiency.

Documentation & Best Practices

• Maintain documentation for data pipelines, architecture, and workflows.
• Follow best practices for data governance, security, and performance optimization.

Qualifications:

• Bachelor's degree in Computer Science, Engineering, Data Engineering, or related field.
• 4–7 years of experience in Big Data / Hadoop development.

Required Skills:

• Strong experience with Hadoop ecosystem tools.
• Hands-on experience with Hive, Impala, HDFS, and Spark.
• Experience working with Cloudera Data Platform (CDP).
• Experience building ETL pipelines and data workflows.
• Knowledge of SQL and big data query optimization.
• Experience with workflow orchestration tools such as Airflow.
• Good understanding of distributed data processing concepts.

Preferred Skills:

• Experience with Python, Scala, or Java for big data development.
• Knowledge of data lake architecture and big data design patterns.
• Familiarity with data governance tools like Ranger and Atlas.
• Experience integrating big data platforms with analytics and BI tools.