Bigdata team member - L3

Ness Group; Hyderabad; Hybrid; INR 2,500,000 - 4,200,000; Full time

Job summary

Ness in Hyderabad is seeking a hands-on Big Data Engineer to design and manage large-scale data platforms with Spark, Hadoop, and cloud-native tools. You will develop batch and streaming pipelines in Python/Scala and optimize data workflows across huge data volumes.

The ideal candidate has strong Spark/Hadoop experience, SQL/NoSQL proficiency, and will collaborate with architects, analysts, and stakeholders to deliver robust data products. Hybrid remote options are available.

Qualifications

• 6-10 years of experience in big data development and operations.
• Strong programming expertise in Python and/or Scala.
• Hands-on with Apache Spark, Hadoop, Cloudera, Hive, and HDFS.
• Proficiency with SQL, NoSQL, Oracle, and scheduling tools (e.g., Autosys).
• Experience in performance tuning, troubleshooting, and distributed systems.
• Familiarity with Git, Jenkins, Ansible, and agile delivery models.

Responsibilities

• Design and develop big data solutions using Apache Spark, Hadoop, and cloud-native tools.
• Write efficient batch and stream processing pipelines in Python and/or Scala.
• Maintain and optimize data workflows and performance across large data volumes.
• Manage source control, versioning, and deployments using Git, Jenkins, and other DevOps tools.
• Conduct performance tuning, system monitoring, and issue resolution.
• Collaborate with architects, analysts, and business stakeholders to align technical delivery with use cases.
• Work with Cloudera Hadoop and manage activities like node configuration, file system maintenance, and backups.
• Follow best practices across software development, code quality, and agile ceremonies.
• Contribute to code reviews, technical documentation, and mentoring junior developers.
• Support and enhance production environments, including disaster recovery planning.

Skills

Python/Scala

Spark

Hadoop

Cloudera/Hive/HDFS

SQL/NoSQL

Git/Jenkins

Agile

Distributed systems

Tools

Autosys

Ansible

Job description

Location: Hyderabad, India, India
Date Created: Oct 8, 2026
Requisition ID: 9431

At Ness, people come first. Here, you'll be part of a vibrant team that values curiosity, innovation, and growth. We work with industry-leading clients on projects that truly make an impact while supporting every team member in carving out their unique career path. With resources for learning, certifications, and hands-on experiences, Ness offers you more than just a job - it's a place where your ideas, ambition, and well-being matter.

The Role

We are looking for a hands-on Big Data Engineer with deep experience in developing and managing large-scale data platforms. The ideal candidate will bring strong Spark, Python/Scala development skills, and solid understanding of the Hadoop ecosystem. You'll work in Agile teams on financial, analytical, and innovation-driven data products - driving performance optimization, new development, and platform operations.

Key Responsibilities

• Design and develop big data solutions using Apache Spark, Hadoop, and cloud-native tools
• Write efficient batch and stream processing pipelines in Python and/or Scala
• Maintain and optimize data workflows and performance across large data volumes
• Manage source control, versioning, and deployments using Git, Jenkins, and other DevOps tools
• Conduct performance tuning, system monitoring, and issue resolution
• Collaborate with architects, analysts, and business stakeholders to align technical delivery with use cases
• Work with Cloudera Hadoop and manage activities like node configuration, file system maintenance, and backups
• Follow best practices across software development, code quality, and agile ceremonies
• Contribute to code reviews, technical documentation, and mentoring junior developers
• Support and enhance production environments, including disaster recovery planning

Qualifications and Skills

Core Skills

• 6-10 years of experience in big data development and operations
• Strong programming expertise in Python and/or Scala
• Hands-on with Apache Spark, Hadoop, Cloudera, Hive, and HDFS
• Proficiency with SQL, NoSQL, Oracle, and scheduling tools (e.g., Autosys)
• Experience in performance tuning, troubleshooting, and distributed systems
• Familiarity with Git, Jenkins, Ansible, and agile delivery models

Nice to Have

• Knowledge of cloud data solutions (AWS, Azure, GCP)
• Familiarity with DevOps and CI/CD practices
• Cloudera Hadoop Admin certification (for hybrid roles involving infra)
• Hands-on development in fast-paced environments
• Led small teams or initiatives within distributed systems projects
• Strong communicator who can work across business and engineering teams

Why Join Us

At Ness, you will work with diverse, talented professionals who are dedicated to making an impact through technology. We encourage applicants of all backgrounds to apply - even if you don't meet every requirement, we'd love to connect with you if this role excites you. We're committed to creating an inclusive workplace that celebrates each team member's unique talents. With flexible remote options, diverse projects, and access to development resources, joining Ness means building a career that's meaningful and impactful.

What to Expect Next

We believe great experiences start with transparency - and that includes our hiring process. Here's what you can typically expect after you hit "Apply":

1. HR Interview - A conversation to get to know you better and align on expectations
2. Technical Interview - A chance to showcase your skills and experience
3. Client Interview - Ensuring mutual fit between you and the client team
4. Offer Stage - If everything aligns, we'll be happy to make it official

While many of our roles involve just 2-3 steps, some client processes may include additional conversations. Regardless of the path, we aim to keep every step transparent and respectful of your time.