Cloud Data Engineer: Scalable Pipelines & Analytics

Cognizant Technology Solutions Corporation; Sydney; On-site; AUD 90,000 - 100,000; Full time

Job summary

Cognizant Technology Solutions Corporation in Australia seeks a results-driven Data Engineer to design, build, and optimize scalable data pipelines across cloud and distributed systems.

You will apply Python, PySpark, and SQL for ingestion, transformation, and analytics, collaborate with cross-functional teams, and contribute to data quality, governance, and cost efficiency across diverse banking and corporate data sources.

Qualifications

• Databricks Certified Data Engineer Associate.
• SQL (Advance) - HackerRank.
• Python (Basic) – HackerRank.

Responsibilities

• Build and maintain scalable data pipelines using cloud services for ingestion, transformation, and storage.
• Develop data products that integrate customer data from multiple sources into unified profiles.
• Create metadata-driven frameworks to ensure data quality, consistency, and governance.
• Write Pyspark, and Spark SQL scripts to perform complex data transformations, ensuring data pipelines are efficient, scalable, and optimized for analytics and reporting.
• Support ETL processes by fixing errors, improving resiliency, and ensuring data integrity across reports and dashboards.
• Monitor data platforms, respond to alerts, and optimize pipelines for efficiency and cost reduction.
• Perform root cause analysis (RCA) for job failures and ensure stability after releases.
• Collaborate with onsite and offshore teams to coordinate tasks, align processes, and deliver projects smoothly.
• Participate in design reviews, implementation walkthroughs, and production deployments.
• Apply governance best practices to maintain security, lineage, and compliance with organizational and regulatory standards.
• Work closely with stakeholders and architects to capture requirements and deliver reliable solutions.
• Use Atlassian tools (Jira, Confluence, Bitbucket) for project management, documentation, and collaboration.
• Lead projects through effective planning, prioritization, and team coordination to ensure successful delivery.

Skills

PySpark

Spark SQL

SAS

Databricks

Java

Python

Power BI

QlikView

Automation

Web scraping

IBM Sterling

Education

Databricks Certified Data Engineer Associate

SQL Advanced

Python Basics

Tools

Hive

Oracle

SSMS

Azure

Databricks

AWS

Bitbucket

GitHub

Jenkins

Control-M

DataBricks Workflows

Spark Logs

Job description

Cognizant Technology Solutions Corporation in Australia seeks a results-driven Data Engineer to design, build, and optimize scalable data pipelines across cloud and distributed systems.

You will apply Python, PySpark, and SQL for ingestion, transformation, and analytics, collaborate with cross-functional teams, and contribute to data quality, governance, and cost efficiency across diverse banking and corporate data sources.