Big Data Engineer

Zorba AI; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Zorba AI in Hyderabad, India, seeks a Big Data Engineer with strong software engineering and distributed data processing skills. You will build scalable data solutions using Java, Apache Spark, Python/Scala, and the Hadoop ecosystem.

Responsibilities include designing data pipelines, integrating new data sources, collaborating with onsite and offshore teams, and driving production deployments while ensuring reliability and performance.

Qualifications

• Hands-on experience with Java and Apache Spark.
• Proficient in Python and/or Scala.
• Experience with Hadoop ecosystem and large-scale data processing.
• Understanding of data ingestion, transformation, and distributed processing.
• Experience designing scalable data pipelines.
• Strong OO programming, design and analysis skills.
• Understanding of secure application development.
• Experience with Agile development methodologies.
• Knowledge of automated QA/testing for Big Data apps.

Responsibilities

• Analyze complex systems and design new data products.
• Develop and integrate new data sources and processing solutions.
• Design scalable distributed data replication strategies.
• Work with large datasets using Hadoop components.
• Develop and deploy data ingestion and transformation pipelines.
• Collaborate with onsite and offshore teams on architecture and development.
• Own product features from development to production deployment.
• Evaluate technologies and prototype data processing improvements.
• Automate development, testing, and deployment processes.

Skills

Java

Apache Spark

Python

Scala

Distributed data processing

Agile methodologies

QA/testing practices

Communication

Security practices

Tools

Hadoop ecosystem

Apache Airflow

Job description

We are looking for an experienced Big Data Engineer with strong software engineering and distributed data processing skills. The ideal candidate should have hands‑on experience with Java, Apache Spark, Python/Scala, and the Hadoop ecosystem to build scalable, reliable data solutions.

Roles & Responsibilities

• Analyze complex systems and collaboratively design new data products and services.
• Develop and integrate new data sources, tools, and data processing solutions.
• Design and implement scalable and reliable distributed data replication strategies.
• Work with large datasets using Hadoop ecosystem components.
• Develop and deploy data ingestion and transformation pipelines.
• Collaborate with onsite and offshore teams on architecture, design, and development.
• Own product features throughout development, testing, and production deployment.
• Evaluate big data technologies and prototype solutions to improve data processing architecture.
• Automate development, testing, and deployment processes.

Required Skills

• Strong hands‑on experience in Java and Apache Spark.
• Good programming knowledge of Python and/or Scala.
• Experience with the Hadoop ecosystem and large‑scale data processing.
• Strong understanding of data ingestion, transformation, and distributed data processing.
• Experience designing scalable and reliable data pipelines.
• Strong object‑oriented programming, design, and analysis skills.
• Understanding of secure application development practices.
• Experience working with Agile development methodologies.
• Understanding of automated QA and testing requirements for Big Data applications.
• Excellent written and verbal communication skills.

Good to Have

• Experience with Apache Airflow.
• Experience with Big Data workflow orchestration and automation.
• Experience mentoring developers and providing technical direction.

Preferred Candidate

Big Data Engineers / Java Spark Developers with strong coding fundamentals and hands‑on distributed data processing experience.

Skills: communication,python,big data,java,hadoop,scala