Lead Data Engineer

Cognizant; Teaneck Township (NJ); On-site; AUD 140,000 - 200,000; Full time

Benefits offered by this job

Career tracks

Education & training

Wellness & benefits

Job summary

Cognizant is seeking an experienced Lead Data Engineer to design, develop, optimize, and implement scalable Big Data solutions, data pipelines, and Azure-based data platforms. You will lead cross-functional teams and drive architectural decisions across design, build, and deployment stages.

The role requires strong leadership, hands-on expertise with Spark/Azure tools, and the ability to translate business needs into robust data engineering solutions.

Qualifications

• Extensive hands-on experience with Spark, PySpark, Scala/Java, Python, SQL, Hive, Kafka, Azure Synapse Analytics, Azure Data Lake Storage, Azure Data Factory, Databricks and distributed data processing frameworks.

Responsibilities

• Lead end-to-end design and delivery of Big Data pipelines, frameworks, reusable components, and data processing solutions as per business and technical requirements.
• Provide technical leadership to developers and engineers, review designs and code, and ensure adherence to engineering best practices.
• Identify, analyse, and resolve complex Big Data performance, scalability, data quality, and production issues independently.
• Collaborate with architects, product owners, business stakeholders, QA, DevOps, and support teams across design, build, testing, deployment, and post-production support.
• Drive defect fixes during QA, UAT, production validation, and warranty phases while ensuring timely delivery and high-quality outcomes

Skills

Technical leadership

Spark

PySpark

Scala/Java

Python

SQL

Hive

Kafka

Azure Synapse Analytics

Azure Data Lake Storage

Azure Data Factory

Databricks

Tools

HDFS

MapReduce

Oozie

Cloudera

Azure Cloud

Job description

Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process services, dedicated to helping the world's leading companies build stronger businesses. Headquartered in Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry and business process expertise, and a global, collaborative workforce that embodies the future of work. Cognizant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing and fastest growing companies in the world.

Our Culture

Your passion, integrity and experience are integral to Cognizant's success. You will join a dynamic and expanding global leader in IT and Business consultancy where you will be valued for who you are. We take pride in our partnership with our clients, so your ability to add value and provide exceptional service to our clients is fundamental to your success. In return, you will be presented with opportunities to develop your career and collaborate with talented colleagues in a supportive, diverse environment.

At Cognizant we recognize that companies that are open and welcoming to a multi-culturally diverse workforce will thrive with fresh perspectives and collaborative knowledge. Cognizant focuses on promoting & increasing gender diversity and providing a workplace which encourages great participation and an equal playing field, where merit and accomplishment are the only criteria for success.

Job Summary

We are seeking an experienced Lead Data Engineer to lead the design, development, optimization, and implementation of scalable Big Data solutions, data pipelines, Azure Synapse Analytics solutions, and Azure Cloud-based data engineering platforms.

Responsibilities

• Lead end-to-end design and delivery of Big Data pipelines, frameworks, reusable components, and data processing solutions as per business and technical requirements.
• Provide technical leadership to developers and engineers, review designs and code, and ensure adherence to engineering best practices.
• Identify, analyse, and resolve complex Big Data performance, scalability, data quality, and production issues independently.
• Collaborate with architects, product owners, business stakeholders, QA, DevOps, and support teams across design, build, testing, deployment, and post-production support.
• Drive defect fixes during QA, UAT, production validation, and warranty phases while ensuring timely delivery and high-quality outcomes

Skills Required

• Strong technical leadership, analytical, problem-solving, and decision-making skills to deliver complex Big Data and Azure Cloud data engineering solutions.
• Extensive hands-on experience with Spark, PySpark, Scala or Java, Python, SQL, Hive, Kafka, Azure Synapse Analytics, Azure Data Lake Storage, Azure Data Factory, Databricks, and distributed data processing frameworks.
• Ability to design scalable batch and streaming data pipelines on Azure Cloud, optimize data workloads, troubleshoot production issues, and guide engineering teams on best practices.
• Strong knowledge of HDFS, MapReduce, Hive, Impala, Tez, Sqoop, Oozie, HBase, Cassandra, MongoDB, Kafka, Spark, Storm, Knox, Ranger, Flume, NiFi, Kerberos, Sentry, Cloudera Manager, Cloudera Navigator, Ambari, and Azure Cloud technologies.
• Independent high-level design ownership and technical leadership across Big Data delivery initiatives.
• Strong communication skills with the ability to influence engineering teams, architects, business stakeholders, QA, DevOps, and production support teams.
• Ability to manage stakeholder expectations, drive delivery governance, mentor team members, and ensure quality engineering outcomes

Benefits

Joining Cognizant will give you the opportunity to learn and collaborate with some of the most talented people in the industry, while having your finger on the pulse of emerging industry trends and working on the cutting edge of technology in your field of expertise.

We Recognize That Our People Perform At Their Best When They Feel Valued As Significant Contributors And That Is Why At Cognizant, Taking Care Of Our Employees Is a Priority

• You can pursue innovative career tracks and opportunities here
• You can enhance your professional development through education and dedicated training
• We’ll give you the skills you need to keep pace with the changing workplace while our compensation, benefits and wellness packages help you stay healthy and plan for the future.