Cloud Data Engineer: Build Scalable Pipelines & Insights

Cognizant; Sydney; On-site; AUD 90,000 - 100,000; Full time

Job summary

Cognizant is seeking a results-driven Data Engineer in Sydney to design, build, and optimize scalable data pipelines across cloud and distributed systems. The role focuses on Python, PySpark, and SQL for ingestion, transformation, and data integration to support analytics and reporting.

You will work with data governance, metadata-driven frameworks, and cost‑effective optimization while collaborating with global teams to deliver reliable data solutions that empower business decision-making.

Qualifications

• Databricks Certified Data Engineer Associate.
• SQL (Advanced) - HackerRank.
• Python (Basic) – HackerRank.

Responsibilities

• Build scalable data pipelines on cloud services for ingestion, transformation, and storage.
• Develop data products integrating customer data from multiple sources into unified profiles.
• Create metadata-driven frameworks to ensure data quality, consistency, and governance.
• Write PySpark and Spark SQL scripts for complex transformations to optimize analytics pipelines.
• Support ETL processes by fixing errors, improving resiliency, and ensuring data integrity across reports and dashboards.
• Monitor data platforms, respond to alerts, and optimize pipelines for efficiency and cost.
• Perform RCA for job failures and ensure stability after releases.
• Collaborate with onsite and offshore teams to coordinate tasks and deliver projects smoothly.
• Participate in design reviews, implementation walkthroughs, and production deployments.
• Apply governance practices to maintain security, lineage, and regulatory compliance.
• Work with stakeholders and architects to capture requirements and deliver reliable solutions.
• Use Jira/Confluence/Bitbucket for project management and documentation.
• Lead projects through planning, prioritization, and coordination for successful delivery.

Skills

PySpark

Spark SQL

SAS

Databricks

Java

Python

Power BI

QlikView

RPA

Web Scraping

IBM Sterling

Education

Databricks Certified Data Engineer Associate

SQL (Advanced)

Python (Basic)

Tools

Azure

Databricks

AWS

Jira

Confluence

Bitbucket

Jenkins

Job description

Cognizant is seeking a results-driven Data Engineer in Sydney to design, build, and optimize scalable data pipelines across cloud and distributed systems. The role focuses on Python, PySpark, and SQL for ingestion, transformation, and data integration to support analytics and reporting.

You will work with data governance, metadata-driven frameworks, and cost‑effective optimization while collaborating with global teams to deliver reliable data solutions that empower business decision-making.