Lead Data Engineer: Azure & Big Data Architect

Cognizant; Teaneck Township (NJ); On-site; AUD 140,000 - 200,000; Full time

Benefits offered by this job

Career tracks

Education & training

Wellness & benefits

Job summary

Cognizant is seeking an experienced Lead Data Engineer to design, develop, optimize, and implement scalable Big Data solutions, data pipelines, and Azure-based data platforms. You will lead cross-functional teams and drive architectural decisions across design, build, and deployment stages.

The role requires strong leadership, hands-on expertise with Spark/Azure tools, and the ability to translate business needs into robust data engineering solutions.

Qualifications

• Extensive hands-on experience with Spark, PySpark, Scala/Java, Python, SQL, Hive, Kafka, Azure Synapse Analytics, Azure Data Lake Storage, Azure Data Factory, Databricks and distributed data processing frameworks.

Responsibilities

• Lead end-to-end design and delivery of Big Data pipelines, frameworks, reusable components, and data processing solutions as per business and technical requirements.
• Provide technical leadership to developers and engineers, review designs and code, and ensure adherence to engineering best practices.
• Identify, analyse, and resolve complex Big Data performance, scalability, data quality, and production issues independently.
• Collaborate with architects, product owners, business stakeholders, QA, DevOps, and support teams across design, build, testing, deployment, and post-production support.
• Drive defect fixes during QA, UAT, production validation, and warranty phases while ensuring timely delivery and high-quality outcomes

Skills

Technical leadership

Spark

PySpark

Scala/Java

Python

SQL

Hive

Kafka

Azure Synapse Analytics

Azure Data Lake Storage

Azure Data Factory

Databricks

Tools

HDFS

MapReduce

Oozie

Cloudera

Azure Cloud

Job description

Cognizant is seeking an experienced Lead Data Engineer to design, develop, optimize, and implement scalable Big Data solutions, data pipelines, and Azure-based data platforms. You will lead cross-functional teams and drive architectural decisions across design, build, and deployment stages.

The role requires strong leadership, hands-on expertise with Spark/Azure tools, and the ability to translate business needs into robust data engineering solutions.