Senior Data Engineer - Lead Scalable Pipelines & Real-Time

Xpt Software Australia Pty; New South Wales; On-site; AUD 140,000 - 180,000; Full time

Job summary

Xpt Software Australia Pty is seeking a Senior Data Engineer to design, build and operate scalable, secure data platforms in Australia. You will own critical data pipelines powering analytics, reporting and real-time insights, leveraging Azure Data Lake Gen2, Synapse and Databricks, and mentoring junior engineers.

You will collaborate with analytics, data science, product and business teams, drive engineering best practices, architecture ownership and trade-off decisions, and contribute to

Qualifications

• Experience with designing and building scalable data platforms.
• Strong knowledge of Azure data services and Databricks.
• Proficient in PySpark/Spark SQL and performance tuning.
• Experience mentoring engineers and communicating trade-offs.
• Collaborates with analytics, data science, product and business teams.

Responsibilities

• Design, build, and operate scalable data pipelines.
• Ensure security and high performance of data platforms.
• Lead technical direction and architecture ownership.
• Mentor junior and mid-level engineers.
• Collaborate across analytics, data science, product and business teams.

Skills

Azure Data Lake Gen2

Synapse

Databricks

PySpark

Spark SQL

Complex SQL

Mentoring

Architect communication

Streaming & Real‑Time Data Processing

Tools

Azure DevOps

GitHub Actions

Job description

Xpt Software Australia Pty is seeking a Senior Data Engineer to design, build and operate scalable, secure data platforms in Australia. You will own critical data pipelines powering analytics, reporting and real-time insights, leveraging Azure Data Lake Gen2, Synapse and Databricks, and mentoring junior engineers.

You will collaborate with analytics, data science, product and business teams, drive engineering best practices, architecture ownership and trade-off decisions, and contribute to