Senior Data Engineer - Azure Spark for Banking

Cognizant Technology Solutions Corporation; Sydney; Sur place; AUD 80 000 - 90 000; Plein temps

Résumé du poste

Cognizant Technology Solutions Corporation is seeking an Experienced Data Engineer with 7+ years in designing and optimizing large-scale data platforms for Banking and Financial Services. Proficiency in Spark, PySpark, Spark SQL, HDFS, ADLS2, Azure Synapse, Python, Hive, and DBMSs, plus DP-203 certification, is required.

Collaboration across SDLC via Agile/Waterfall is expected. The role involves building scalable pipelines, tuning performance, and leading critical data initiatives with

Qualifications

• 7+ years of experience in designing, developing, and optimizing large-scale data engineering solutions.

Responsabilités

• Collaborate with cross-functional teams to analyze, design, and implement scalable data engineering solutions.
• Design, develop, and optimize enterprise-grade data processing pipelines using PySpark and Spark SQL.
• Build and maintain scalable data ingestion, transformation, and aggregation frameworks leveraging Spark DataFrames, Hive, HDFS, and ADLS2.
• Perform Spark performance tuning and optimization to improve resource utilization and processing efficiency.
• Develop complex Spark SQL and HiveQL queries to support business-critical data processing requirements.
• Design and implement robust data transformation logic while adhering to coding standards and best practices.
• Investigate, troubleshoot, and resolve production issues through detailed log analysis and root cause identification.
• Drive operational excellence by implementing proactive monitoring and reliability improvements.
• Configure, schedule, and manage enterprise data workflows using BMC Control-M.
• Manage source code versioning and collaboration through Git and Bitbucket.
• Create, review, and implement Change Requests (CRQs) in accordance with organizational governance processes.
• Maintain comprehensive technical documentation, deployment procedures, and operational support artifacts.
• Support production releases and platform upgrade activities while ensuring service continuity.
• Mentor junior developers and provide technical guidance to team members.

Connaissances

Apache Spark

PySpark

Spark SQL

HDFS

ADLS2

Azure Synapse Analytics

ServiceNow

Control-M

Bitbucket

Jenkins

CI/CD Pipeline

Formation

DP-203 Data Engineer

Databricks

AI Security & Governance

Outils

Git

Bitbucket

Jenkins

CI/CD

Description du poste

Cognizant Technology Solutions Corporation is seeking an Experienced Data Engineer with 7+ years in designing and optimizing large-scale data platforms for Banking and Financial Services. Proficiency in Spark, PySpark, Spark SQL, HDFS, ADLS2, Azure Synapse, Python, Hive, and DBMSs, plus DP-203 certification, is required.

Collaboration across SDLC via Agile/Waterfall is expected. The role involves building scalable pipelines, tuning performance, and leading critical data initiatives with