Bigdata team member - L3

Ness Digital Engineering; Hyderabad; Remote; INR 1,800,000 - 2,400,000; Full time

Job summary

Ness Digital Engineering is seeking a hands-on Big Data Engineer to design and manage large-scale data platforms. You’ll leverage Spark, Hadoop, and cloud-native tools, coding in Python and/or Scala, and collaborate in Agile teams to optimize performance and delivery.

You will work with Cloudera Hadoop, Hive, HDFS, and modern data workflows, contributing to platform operations and production support across financial, analytical, and innovation-driven data products.

Qualifications

• 6-10 years of experience in big data development and operations.
• Strong programming expertise in Python and/or Scala.
• Hands-on with Apache Spark, Hadoop, Cloudera, Hive, and HDFS.
• Proficiency with SQL, NoSQL, Oracle, and scheduling tools (e.g., Autosys).
• Experience in performance tuning, troubleshooting, and distributed systems.
• Familiarity with Git, Jenkins, Ansible, and agile delivery models.

Responsibilities

• Design and develop big data solutions using Apache Spark, Hadoop, and cloud-native tools
• Write efficient batch and stream processing pipelines in Python and/or Scala
• Maintain and optimize data workflows and performance across large data volumes
• Manage source control, versioning, and deployments using Git, Jenkins, and other DevOps tools
• Conduct performance tuning, system monitoring, and issue resolution
• Collaborate with architects, analysts, and business stakeholders to align technical delivery with use cases
• Work with Cloudera Hadoop and manage activities like node configuration, file system maintenance, and backups
• Follow best practices across software development, code quality, and agile ceremonies
• Contribute to code reviews, technical documentation, and mentoring junior developers
• Support and enhance production environments, including disaster recovery planning

Skills

Big data development

Python

Scala

Spark

Hadoop

Cloudera

Hive

HDFS

SQL

NoSQL

Git

Jenkins

Ansible

Agile delivery

Tools

Autosys

AWS

Azure

GCP

Job description

Life at Ness

At Ness, people come first. Here, you'll be part of a vibrant team that values curiosity, innovation, and growth. We work with industry-leading clients on projects that truly make an impact while supporting every team member in carving out their unique career path. With resources for learning, certifications, and hands‑on experiences, Ness offers you more than just a job-it’s a place where your ideas, ambition, and well‑being matter.

Description

Life at Ness

At Ness, people come first. Here, you'll be part of a vibrant team that values curiosity, innovation, and growth. We work with industry-leading clients on projects that truly make an impact while supporting every team member in carving out their unique career path. With resources for learning, certifications, and hands‑on experiences, Ness offers you more than just a job-it’s a place where your ideas, ambition, and well‑being matter.

The Role

We are looking for a hands‑on Big Data Engineer with deep experience in developing and managing large‑scale data platforms. The ideal candidate will bring strong Spark, Python/Scala development skills, and solid understanding of the Hadoop ecosystem. You’ll work in Agile teams on financial, analytical, and innovation-driven data products- driving performance optimization, new development, and platform operations.

Key Responsibilities

• Design and develop big data solutions using Apache Spark, Hadoop, and cloud‑native tools


• Write efficient batch and stream processing pipelines in Python and/or Scala


• Maintain and optimize data workflows and performance across large data volumes


• Manage source control, versioning, and deployments using Git, Jenkins, and other DevOps tools


• Conduct performance tuning, system monitoring, and issue resolution


• Collaborate with architects, analysts, and business stakeholders to align technical delivery with use cases


• Work with Cloudera Hadoop and manage activities like node configuration, file system maintenance, and backups


• Follow best practices across software development, code quality, and agile ceremonies


• Contribute to code reviews, technical documentation, and mentoring junior developers


• Support and enhance production environments, including disaster recovery planning

Qualifications And Skills

Core Skills

• 6-10 years of experience in big data development and operations


• Strong programming expertise in Python and/or Scala


• Hands‑on with Apache Spark, Hadoop, Cloudera, Hive, and HDFS


• Proficiency with SQL, NoSQL, Oracle, and scheduling tools (e.g., Autosys)


• Experience in performance tuning, troubleshooting, and distributed systems


• Familiarity with Git, Jenkins, Ansible, and agile delivery models

Nice to Have

• Java development background


• Financial domain experience


• Knowledge of cloud data solutions (AWS, Azure, GCP)


• Familiarity with DevOps and CI/CD practices


• Cloudera Hadoop Admin certification (for hybrid roles involving infra)


• Hands‑on development in fast‑paced environments


• Led small teams or initiatives within distributed systems projects


• Supported large‑scale production platforms


• Strong communicator who can work across business and engineering teams

Why Join Us

At Ness, you will work with diverse, talented professionals who are dedicated to making an impact through technology. We encourage applicants of all backgrounds to apply—even if you don’t meet every requirement, we’d love to connect with you if this role excites you. We’re committed to creating an inclusive workplace that celebrates each team member’s unique talents. With flexible remote options, diverse projects, and access to development resources, joining Ness means building a career that’s meaningful and impactful.

What To Expect Next

• HR Interview - A conversation to get to know you better and align on expectations


• Technical Interview - A chance to showcase your skills and experience


• Client Interview - Ensuring mutual fit between you and the client team


• Offer Stage - If everything aligns, we’ll be happy to make it official

While many of our roles involve just 2-3 steps, some client processes may include additional conversations. Regardless of the path, we aim to keep every step transparent and respectful of your time.

Ready to Start Your Journey?