Azure Data Engineer – PySpark, Synapse & Data Lakes

Cognizant; Sydney; On-site; AUD 150,000 - 210,000; Full time

Benefits offered by this job

Career development opportunities

Training and certifications

Health insurance and wellness programs

Job summary

Cognizant in Sydney is seeking an experienced Azure Data Engineer to design, develop, and optimize scalable data pipelines using PySpark, Azure Synapse Analytics, and modern cloud data practices.

The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines, distributed data processing frameworks, and enterprise-grade analytics solutions across Microsoft Azure and Big Data platforms.

Qualifications

• Proven experience in designing, developing and supporting production-grade data pipelines.
• Hands-on expertise with PySpark, SQL and Azure Synapse Analytics.
• Experience with big data ecosystems and cloud data services.

Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark, Spark SQL, Python, SQL, and Azure Synapse Analytics.
• Build and optimize ETL/ELT workflows for large-volume data.
• Develop and support Azure Synapse pipelines, notebooks, SQL scripts, stored procedures, and data transformation logic.
• Work with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Databricks, Azure SQL Database, and related services.
• Develop and support Big Data solutions using Spark, Hadoop, HDFS, Hive, Kafka, and related components.
• Translate requirements into detailed technical designs and reusable data components.
• Implement ingestion, transformation, validation, reconciliation, and publishing processes.
• Tune PySpark and SQL workloads; optimize joins, partitioning, and execution plans.
• Collaborate with architects, analysts, QA, DevOps, platform teams, and stakeholders.
• Ensure governance, security, lineage, audit, and access control compliance.
• Mentor junior data engineers and promote best practices.

Skills

PySpark

Azure Synapse

Python

SQL

Big Data

ETL/ELT

Data pipelines

Cloud data services

Mentoring

Job description

Cognizant in Sydney is seeking an experienced Azure Data Engineer to design, develop, and optimize scalable data pipelines using PySpark, Azure Synapse Analytics, and modern cloud data practices.

The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines, distributed data processing frameworks, and enterprise-grade analytics solutions across Microsoft Azure and Big Data platforms.