Senior Data Engineer — Spark, Azure & Banking

Cognizant Technology Solutions Corporation

Sydney

In loco

AUD 80.000 - 90.000

Tempo pieno

2 giorni fa

Candidati tra i primi

Aumenta le possibilità di ricevere una risposta da questo datore di lavoro

Il tuo curriculum e una lettera di presentazione personalizzati, che corrispondono esattamente a ciò che sta cercando.

Supera i filtri ATS

Descrizione del lavoro

Cognizant Technology Solutions Corporation in Sydney, Australia seeks an Experienced Data Engineer with 7+ years in designing, developing, and optimizing large-scale data pipelines for Banking and Financial Services. Proficient in Spark, PySpark, SparkSQL, HDFS, ADLS2, Azure Synapse, Python, and HBase, with DP-203 Azure Data Engineer Associate certification.

Responsibilities include building scalable ingestion, transformation, and processing frameworks, performance tuning, and collaborating with

Competenze

Experience building and optimizing large-scale data pipelines in Banking/Financial Services.
Strong task understanding of Spark-based data processing, ingestion, and transformation.

Mansioni

Collaborate with cross-functional teams to analyze, design, and implement scalable data engineering solutions.
Design, develop, and optimize enterprise-grade data processing pipelines using PySpark and Spark SQL.
Build and maintain scalable data ingestion, transformation, and aggregation frameworks leveraging Spark DataFrames, Hive, HDFS, and ADLS2.
Perform Spark performance tuning to improve resource utilization and processing efficiency.
Develop complex Spark SQL/HiveQL queries to support data processing requirements.
Design and implement robust data transformation logic with coding best practices.
Investigate, troubleshoot, and resolve production issues via log analysis and root cause ID.
Drive operational excellence with proactive monitoring and reliability improvements.
Configure, schedule, and manage enterprise data workflows using BMC Control-M.
Manage source code versioning through Git/Bitbucket.
Create, review, and implement Change Requests per governance processes.
Maintain technical docs, deployment procedures, and operational artifacts.
Support production releases and platform upgrades ensuring service continuity.
Mentor junior developers and provide technical guidance.