Big Data Hadoop Engineer

Wipro; Maharashtra; On-site; INR 2,800,000 - 4,200,000; Full time

Job summary

Wipro Technologies in India is seeking a seasoned Technical Lead for Big Data and Hadoop environments to drive development of large-scale data pipelines, ensuring performance, scalability and data integrity.

You will mentor peers, collaborate with architects and product owners, apply Spark, Databricks, Kafka, NiFi and Airflow, and define robust replication and testing strategies to meet SLAs and business objectives.

Qualifications

• Minimum 5 years’ experience designing, building and operating robust end-to-end data pipelines.
• Bachelor’s degree in a technical or quantitative discipline (CS/Engineering/Math/Stats).
• Strong communication skills for cross‑functional collaboration in geo-distributed environments.
• Deep hands-on experience with Spark and cloud-based data solutions such as Databricks.
• Proficiency in SQL, Python and Scala for data engineering tasks.

Responsibilities

• Lead design, implementation, automation and maintenance of large-scale ETL pipelines to meet performance targets.
• Mentor and grow other data engineers, fostering collaboration and knowledge sharing.
• Serve as technical authority on data engineering best practices and troubleshooting.
• Promote code reviews, data validation and continuous testing standards.
• Deliver resilient data pipelines with defined latency and reliability SLAs.
• Collaborate with architects, engineers, and product owners to align data needs with business goals.
• Work with upstream data teams to ensure data quality and availability for downstream consumers.
• Develop data pipelines using Spark, Databricks, Kafka, NiFi and Airflow for complex processing tasks.
• Drive automation, monitoring and observability improvements in data operations.
• Identify bottlenecks and optimize using profiling tools to maintain pipeline performance.

Skills

Data pipelines

Team leadership

Agile methods

Test driven development

Root cause analysis

Communication

Education

Bachelor's degree in CS/Engineering

Tools

SQL

Python

Scala

Apache Spark

Databricks

Airflow

Kafka

NiFi

Hadoop Cloudera

Job description

Job description:

Job Description

Role Purpose and Overview

This role is designed for a seasoned Technical Lead specializing in Big Data and Hadoop environments. The primary focus is to drive the development and optimization of complex data pipelines, ensuring high performance, scalability, and data integrity. You will be responsible for developing, reviewing, and enhancing module-level code, designing robust replication strategies to promote data reliability, and conducting thorough root cause analyses to resolve recurring data issues. Collaboration is key, as you will work closely with both internal teams and external partners to deliver thoroughly documented, well-architected solutions that align with overarching business goals and technical standards.

This role provides a unique opportunity to lead and influence significant data engineering projects and to mentor peers, contributing to a culture of continuous improvement and shared technical excellence. You will be instrumental in enabling the organization’s data-driven strategies through innovative application of industry-leading big data technologies and architectural best practices.

Key Responsibilities

• Lead comprehensive projects encompassing the design, implementation, automation, and ongoing maintenance of large-scale ETL pipelines, ensuring they meet stringent product requirements and performance benchmarks.
• Support and mentor other data engineers, facilitating their integration and growth within the team, while nurturing a collaborative, knowledge-sharing environment.
• Serve as a technical authority and subject matter expert, providing guidance in data engineering best practices and troubleshooting complex data-centric issues.
• Adopt and advocate for industry standards such as source control discipline, comprehensive code reviews, rigorous data validation, and continuous testing methodologies.
• Deliver resilient and efficient data pipelines that meet or exceed defined latency and reliability service level agreements (SLAs).
• Engage proactively with architects, software engineers, and product owners to clarify data needs and ensure alignment between technical delivery and business objectives.
• Collaborate closely with upstream data teams and governance bodies to uphold data quality standards and ensure data availability for downstream consumers.
• Develop and enhance data pipelines utilizing a variety of advanced big data technologies including Apache Spark, AWS EMR, Databricks, Kafka, NiFi, and Airflow to solve complex processing challenges.
• Lead initiatives aimed at improving automation, monitoring, and observability within data operations to promote operational excellence.
• Identify performance bottlenecks and implement optimizations using profiling tools and advanced features within technologies like Databricks to maintain cutting-edge pipeline performance.
• Continuously evaluate emerging technologies and infrastructural enhancements to refine and advance the data ecosystem.

Experience & Skills Required

• Minimum of 5 years’ experience designing, building, and operating robust end-to-end data pipelines with a strong emphasis on data accuracy, integrity, and quality management.
• Comprehensive understanding of data engineering methodologies and firm grounding in software engineering principles. Candidates must possess a bachelor’s degree in a technical or quantitative discipline such as Computer Science, Engineering, Mathematics, or Statistics; advanced degrees are highly valued.
• Exceptional communication skills — both verbal and written — and proven ability to foster effective collaboration and teamwork in dynamic, matrixed, and geo-distributed environments.
• Proven adaptability and eagerness to embrace challenging projects, learning opportunities, and cross-functional teamwork across diverse teams and locations.
• Deep familiarity with agile methodologies and test-driven development, ensuring development agility and product quality.
• Extensive hands‑on experience with big data platforms like Apache Spark and cloud‑based data solutions such as Databricks.
• Strong proficiency in advanced programming and scripting languages including SQL, Python, and Scala.
• Sound knowledge of data pipeline architecture (ETL/ELT), data modeling techniques including star schema designs, and data warehousing concepts.
• Expertise in orchestration and streaming technologies such as Apache Airflow, Kafka, and NiFi to enable high‑throughput real‑time data flows.

Mandatory Skills and Qualifications

• Proven expertise in Hadoop Cloudera ecosystems, including design, deployment, and troubleshooting within enterprise environments.
• Experience working with distributed computing frameworks and managing data storage solutions optimized for big data workloads.
• Strong analytical skills to perform root cause analyses and implement sustainable solutions for complex data issues.
• Demonstrated ability to lead technical teams delivering high‑impact projects on time and to specification, with a commitment to continuous delivery and improvement.
• Commitment to professional growth and staying current with emerging trends in big data and cloud data technologies.

About Wipro Technologies

At Wipro Technologies, we are pioneering the future of digital and data transformation across multiple industries. As a leading end‑to‑end digital services provider, we are committed to empowering our workforce to design their own reinvention and build the next generation of technology solutions. Our culture fosters bold ambitions supported by collaboration, innovation, and purpose‑driven work.

We believe in cultivating an inclusive environment where diverse perspectives thrive, fueling creativity and outstanding results. If you are passionate about evolving your career while contributing to impactful projects that harness cutting‑edge technologies like Hadoop Cloudera, Spark, and cloud platforms, then Wipro is the place for you.

Join us and become part of a team dedicated to excellence, continuous learning, and driving transformational change across the technology landscape. Together, we create meaningful impact, powering businesses with data‑powered insights and intelligent solutions.

Experience Required: 5-8 Years

Mandatory Skills: Hadoop Cloudera

Mandatory Skills: Hadoop Cloudera .

Experience: 5-8 Years.

Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention.