Senior Data Engineer - Azure Databricks & Spark

Renaissance Infosystems Pte Ltd; Sydney; On-site; AUD 120,000 - 180,000; Full time

Job summary

Renaissance Infosystems Pte Ltd in Sydney is seeking a Data Engineer with 6–12 years of experience to design, build, and optimize high-throughput data pipelines on Azure Databricks using Spark (Scala) and SQL. This permanent role is based in Sydney and requires hands-on expertise in data engineering.

You will implement medallion architecture with Delta Lake, develop real-time and near real-time integrations, and oversee CI/CD practices with GitHub/Azure DevOps, ensuring robust data workflows and

Qualifications

• Hands-on developer to design, build, and optimize high throughput data pipelines on Azure Databricks using Spark (Scala) and SQL.
• Design and develop scalable ETL/ELT pipelines in Azure Databricks following medallion (Bronze/Silver/Gold) and Delta Lake best practices.
• Proven expertise in real-time and near real-time integrations and CI/CD with GitHub, Azure DevOps, and observability.

Skills

Azure Databricks

Scala Spark

SQL

Python

Tools

GitHub

Confluence

Jira

Jenkins

Azure DevOps

Delta Lake

Job description

Renaissance Infosystems Pte Ltd in Sydney is seeking a Data Engineer with 6–12 years of experience to design, build, and optimize high-throughput data pipelines on Azure Databricks using Spark (Scala) and SQL. This permanent role is based in Sydney and requires hands-on expertise in data engineering.

You will implement medallion architecture with Delta Lake, develop real-time and near real-time integrations, and oversee CI/CD practices with GitHub/Azure DevOps, ensuring robust data workflows and