Senior Azure Data Engineer | Spark, ADLS2 & Databricks

Cognizant Technology Solutions Corporation; Sydney; Hybrid; AUD 80,000 - 90,000; Full time

Job summary

Cognizant Technology Solutions Corporation in Sydney, Australia, is seeking an experienced Data Engineer with 7+ years’ experience to design, develop and optimize large-scale data solutions for Banking and Financial Services. The role emphasizes Spark, PySpark, Spark SQL, and Azure data services to deliver scalable pipelines.

You will collaborate with global teams, mentor junior engineers, and drive production deployments while ensuring data security and governance.

Qualifications

• Experience designing large-scale data pipelines in Banking/Finance.
• Proficient with Spark-based ETL and data processing.
• Strong SDLC knowledge and Agile/Waterfall.
• Production support and performance tuning experience.
• Ability to mentor juniors.

Responsibilities

• Collaborate with cross-functional teams to design scalable data pipelines.
• Develop and optimize data ingestion and processing with PySpark.
• Build pipelines using Spark DataFrames, Hive, HDFS, and ADLS2.
• Tune Spark jobs for performance and resource efficiency.
• Write complex Spark SQL and HiveQL queries for business needs.
• Implement robust data transformation logic and coding standards.
• Troubleshoot production issues via log analysis and RCA.
• Enhance monitoring and reliability of data workflows with Control-M.
• Manage source control with Git/Bitbucket and CRQ governance.
• Document deployment procedures and operational support artifacts.
• Support platform upgrades and production releases.
• Mentor junior developers and provide technical guidance.

Skills

Apache Spark

PySpark

Spark SQL

HDFS

ADLS2

Azure Synapse Analytics

Python

HiveQL

Spark DataFrames

Production Support

Education

DP-203 Data Engineering

Databricks

IBM AI

Cognizant Certification – Prompt and Context Engineering

AI Security & Governance

Tools

Bitbucket

Jenkins

CI/CD Pipeline

Git

ServiceNow

Control-M

HBase

Job description

Cognizant Technology Solutions Corporation in Sydney, Australia, is seeking an experienced Data Engineer with 7+ years’ experience to design, develop and optimize large-scale data solutions for Banking and Financial Services. The role emphasizes Spark, PySpark, Spark SQL, and Azure data services to deliver scalable pipelines.

You will collaborate with global teams, mentor junior engineers, and drive production deployments while ensuring data security and governance.