Senior Data Engineer — Databricks, Medallion Modeling

EPAM Systems Inc

Sydney

On-site

AUD 140,000 - 190,000

Full time

32 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Great Place To Work-Certified
Glassdoor Top 100 Best Places to Work
Most Loved Workplace (Newsweek 2021-23
Voice-driven ideas culture

Job summary

EPAM Systems Inc. seeks a Senior Data Engineer to model large-scale data across Bronze, Silver and Gold layers using medallion architecture.

You will build Databricks pipelines with Delta Lake and Delta Live Tables, develop Spark workloads in PySpark and Spark SQL, and implement streaming with Structured Streaming and Kafka. You’ll collaborate with stakeholders to deliver governed data products and strengthen data quality and lineage.

Qualifications

  • Data modeling background across conceptual, logical and physical models using dimensional, Data Vault or similar approaches.
  • Databricks experience, including Delta Lake plus governance concepts such as Unity Catalog.
  • Apache Spark skill sets using PySpark and Spark SQL in production environments.
  • Streaming data experience using Spark Structured Streaming, Kafka or similar technologies.
  • Python capability for pipeline development and orchestration patterns.
  • Advanced SQL skill sets for transformation logic, validation and performance tuning.
  • Data governance experience covering quality, lineage, controls and well-defined data contracts.
  • AI-assisted development tooling exposure, with Claude preferred, to accelerate build, testing and documentation.

Responsibilities

  • Design trusted Bronze, Silver and Gold layers using medallion architecture principles.
  • Model large-scale batch and streaming datasets into reusable, governed data models for reporting use cases.
  • Build Databricks pipelines using Delta Lake, Delta Live Tables or Lakeflow aligned to quality and lineage needs.
  • Develop Spark workloads using PySpark and Spark SQL for performance, reliability and maintainability.
  • Implement streaming ingestion and processing using Spark Structured Streaming and Kafka or similar.
  • Partner with business and regulatory stakeholders to translate reporting needs into data products and model definitions.
  • Strengthen governance with cataloging, lineage and data quality checks using Unity Catalog or equivalent patterns.
  • Contribute to CI/CD for data pipelines, Git-based workflows, testing and release practices.

Skills

Data modeling
Databricks
Spark (PySpark)
Streaming
Python
SQL
Data governance
AI-assisted tooling

Tools

Delta Lake
Delta Live Tables
Lakeflow
Unity Catalog
Kafka

Job description

EPAM Systems Inc. seeks a Senior Data Engineer to model large-scale data across Bronze, Silver and Gold layers using medallion architecture.

You will build Databricks pipelines with Delta Lake and Delta Live Tables, develop Spark workloads in PySpark and Spark SQL, and implement streaming with Structured Streaming and Kafka. You’ll collaborate with stakeholders to deliver governed data products and strengthen data quality and lineage.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Databricks Data Engineer - Medallion Data Platforms
Senior Databricks Data Engineer - Medallion Data Platforms

EPAM Systems Inc • Australia

On-site
AUD 150,000 - 200,000
Senior Databricks Data Engineer – Medallion Architecture
Senior Databricks Data Engineer – Medallion Architecture

EPAM Systems Inc • Sydney

On-site
AUD 150,000 - 190,000
Senior Data Engineer (Databricks, Data Modeling)
Senior Data Engineer (Databricks, Data Modeling)

EPAM Systems • Australia

On-site
AUD 150,000 - 200,000
Senior Data Engineer (Databricks, Data Modeling)
Senior Data Engineer (Databricks, Data Modeling)

EPAM Systems Inc • Sydney

On-site
AUD 140,000 - 190,000
Great Place To Work-Certified
Glassdoor Top 100 Best Places to Work
Most Loved Workplace (Newsweek 2021-23
+1
Senior Data Engineer
Senior Data Engineer

DX1 • Melbourne

On-site
AUD 160,000 - 230,000
Senior Data Engineer (Databricks, Data Modeling)
Senior Data Engineer (Databricks, Data Modeling)

EPAM Systems • Sydney

On-site
AUD 150,000 - 190,000
Senior Data Engineer
Senior Data Engineer

ALOIS Australia • Sydney

On-site
AUD 180,000 - 230,000
Senior Data Engineer - Cloud Data Platform & ML Pipelines
Senior Data Engineer - Cloud Data Platform & ML Pipelines

Compare The Market • Brisbane

Hybrid
AUD 130,000 - 180,000
Vibrant and social community
Flexible work – one day WFH
‘ME’ leave and ‘Volunteer Day’ leave
+3
Hybrid Data Engineer - Databricks, Delta Lake & PII
Hybrid Data Engineer - Databricks, Delta Lake & PII

Myndshift Group • Queensland

Hybrid
AUD 122,000 - 150,000
Hybrid work up to 2 days from home
Superannuation up to 15.25%
Leave loading 17.5%
+3
Data Engineer: Pipelines & Data Modeling (AWS)
Data Engineer: Pipelines & Data Modeling (AWS)

Lion • Sydney

On-site
AUD 120,000 - 180,000
Top locations